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Abstrac t 


This  thesis  provides  a  foundation  for  future  research 
on  the  subject  of  readability  formulas  and  their  use  with 
Air  Force  publications.  It  ex  am  i  n  e  s  readability’s 
historical  roots  and  studies  per  t  a  i  n  i  tig'  to  the  topic  of 
readability  and  its  application  to  the  United  States 
Military  in  general,  and  the  United  States  Air  Force  in 
part icular. 

This  presentation  also  describes  and  analyzes  Air  For:.'!' 
policies  relative  to  the  readability  of  its  publications. 

The  results  of  this  analysis  indicate  that  the  current 
procedures,  employed  by  the  Air  Force,  inadequately  address 
the  readability  of  publications.  The  specific  problems  that 
were  identified  fell  under  three  areas. 

The  first  area  addresses  the  Air  Force's  decision  to 
u  Technical  Order..  t  r>  a  ninth  reading-  level.  This 

decision  is  unsubstantiated  on  either  economic  or  efficiency 
grounds . 

The  set' oiid  problem  uuiicc.tc  ‘he  ne^r-tionaMe 
methodology  used  by  the*  Air  Force  to  verify  that,  the  desired 
reading  grade  level  has  indeed  been  achieved.  The1  problem 
revolves  around  the  use  and  misuse  of  the  Kincaid 


readab i  1  i t  y  formula. 


Tin-  f  :  no  .  .if  a  f  <  > r  <->t  i.-orn  is  f  hi-  \  i  F-  rre’s  :<•••  i  ^  i  n 
to  H'l'omnu-ii'l  e-  i  n  -g  As  sue  i  at  ion  F.uropeene  Des  f  ms!  rue  i  a;-s 

Dr  Ma  t  i'i’i  >-■  1  Ac  rospa  t  i  a  !  (  AF.CMA  S  i  mp  1  :  f  i  ed  Ena,'  1  i  sh _ Guide  f  <  ■  i- 

t  he  Pnparat  ion  of  Aircraft  Maintenance  Documentat  inn  i  n  _  t  ht- 
Ai-Tospaor  Maintenance  Language)  as  an  aid  for  orortue  i  nif 
simp!  if  iod  t  cchn  i  ca  1  a  r  i  t  inu;.  This  is  a  problem  because 
\F.fM  \  lias  iie\<  r  !>e'-n  emj ■  i  r  i  c a  1  !  y  teste,!. 
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READABILITY  FORMLLAS 


AND  AIR  FORCE  PUBLICATIONS : A 
REVIEW  OF  THEIR  DEVELOPMENT  AND 
AN  E.X.VM  I  NAT  ION  OF  THEIR  APPLICATION 


1  .  O  v  e  r  v  i  e  w 


Genera  I  J  s  sues 

The  United  States  Air  Force  genera!  1  y  ree  ru  i  t  s  h  i  «,’h 
scnuo  1  gradual  es  to  be  member:;  of  its  enlisted  {'..roes  . 

These  individuals  quite  natural  1 y ,  possess  varied  aptitudes 
ar.d  ali  i  1  i  f  ies;  however ,  upon  entering'  the'  service,  t  hey  are 
required  to  be  able  to  read  and  understand  the  numerous 
nq'ola!  ions,  technical  orders,  and  other  government 
documents  which  are  pert  inent  to  their  dut  ies.  Therefore, 
these  pub  1  i  cat  ions  must  be  written  to  a  1  eve  1  eommensu  -*at  <• 

.  i  t  h  the  reading'  si. ills  of  the  intended  audience.  If  they 
are  not  written  to  such  a  level,  t  here  is  the  possibility  of 
:  'nendou.-,  damage  to  equipment  ,  injury  to  personnel  ,  and 

[in  s  s  i  b  1  \  fatalities  (Kincaid  and  others,  1  9  f>  7  :  1  ;  .J  •  ,hn  «.  <  ■  n  ,  K  . 
and  ol  tiers,  1972:7;  Hooke  and  others,  19  7  0:5). 

Because  vital  informat  ion  is  presented  in  these 
publications  it  is  critically  important  that  I  hey  be 
readable;  but  are  they?  If  l  h.-j  are,  *e  can  all  breathe  a 
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i'o!  lee  i  i  vt-  sigh  of  relief,  browing  that  the  servin'  men  and 


vo  men  ,  vho  are  working'  on  our  p  1  anes  ,  m  i  s  s  i  les  ,  radars  ,  an  : 
other  vital  systems  for  the  defense  of  our  country,  are 
being'  given  readable  and  understandable  Technical  Orders  an 
maintenance  documents.  If,  on  the  other  hand,  the  material 
presented  to  the  service  member  are  not  in  a  readab  1  •  form , 
the  potent  ial  for  loss  is  high.  But  aha t  exact  |y 
cons t i t  ut  es  a  readab 1 e  pub  1  i cat i on? 

Thomas  Huckin  defines  writing  as  readable  "to  the 
err  <>ut  that  its  meaning*  can  be  easily  and  quickly 
comprehended  for  an  intended  purpose  by  an  intended  rea nr 
operating  under  normal  conditions  of  alertness,  mot  ivat  ‘  •  ■ :  i . 
t  i  me -pres  sure"  (Huckin,  198:1:91).  In  general,  readab  i  !  i  t  y 
is  measured  according  to  the  grade  level  required  for 
understanding.  Thus  material  with  a  readability  level  .f  :■ 
should  be  able  to  be  understood  by  a  reader  vho  possesses  i 


read i ng 

ab  i  1 

i  t  y  a  t. 

the  9th  grade 

level  .  Norm a  1  1 y 

the 

me t  hodo 1 

1  ogy 

emp 1 oy  t 

■*d  to  test  this 

r e 1  a t  ionship  is 

i  n 

i  !*, 

form  of  readability  formulas. 

K 1  are ,  in  his  1963  book  The  Measurement  of  Readab j  lit; 
states  that  a  readab i 1 ity  formula  should  serve  as  a 
predict  i vp  device  which  would  provide  a  quant  it  at  ive, 
objective  estimate*  as  to  the*  difficulty  level  of  a  piece  of 
writ ing.  He  adds  that  these  formulas  should  be  broad  v noug 
in  scope  so  that  they  could  provide  est i mates  over  a  range 
of  applicability  and  difficulty  (Klare,  1963:33-31). 


To  develop  this  estimate,  readability  formulas  in 
general  use  quantifiable  syllable,  word,  sentence,  or 
paragraph  factors  as  determ'nants  of  a  text’s  reading  level. 
These  factors,  although  employed  differently  by  different 
researchers,  are  used  with  the  ultimate  goal  of  developing  a 
mathematical  technique  for  determining  a  text’s  readabili’y 
level.  Specifically  the  United  States  Air  Force  uses  the 
Kincaid  readability  formula  to  assess  the  readability  of  its 
Technical  Orders  (for  additional  information  on  this  subject 
see  Appendix  C',  and  the  research  of  Kincaid  and  o'hers 
discussed  in  chapter  4). 

In  this  paper  the  subject  of  readability  and 
readability  formulas  and  their  relationship  to  Air  Force' 
publications  will  be  explored  from  the  following 
perspective:  first,  readability’s  historical  roots  will  be 

examined.  Included  in  this  evaluation  will  be  a  description 
of  significant  readability  formulas.  Next,  studies 
per  t.  a  i  n  i  ng  to  the  topic  of  readah  i  I  i  ty  and  i  t  s  app  1  i  ca  t  i  on 
to  the  United  States  Military  in  general,  and  the  United 
States  Air  Force  in  particular,  will  be  reviewed.  Finally, 
conclusions  will  be  drawn  concerning  the  applicability  of 
using  readability  formulas  when  writing  military 
pub  1  i ca t ions  . 
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Pr  oh  I  em  Statement 

The  United  States  Air  Force’s  military  and  civilian 
members  are  given  the  responsibility  for  writing 
regulations,  training  materials,  and  other  Air  Force 
publications  with  little  guidance  on  how  to  complete  their 
tasks.  Fortunately  most  of  the  individuals  assigned  "he 
responsibility  for  writing  these  publications  have  the 
technical  skills  required  to  know  what  information  should  b 
included  in  them.  Unfortunately,  however,  these  same  peopl 
may  not  have  the  requisite  writing  ability  necessary  to 
effectively  transfer  their  knowledge  to  their  audience  by 
producing  a  readable  document. 

One  likely  scenario  has  the  author  seeking  assistance 
from  coworkers  on  how  to  best  accomplish  the  tasking.  From 
these  sources  the  author  could  be  given  such  advice  as: 
"write  it  just  like  the  old  one,"  or  "go  to  the  Air  Force  0 
2  (a  document  listing  all  Air  Force  Regulations);  I  am  sure 
they  must  have  information  on  the  subject." 

If,  on  the  one  hand,  our  author  decides  to  write  the 
new  document  in  the  same  format  as  the  old  one,  any 
readability  errors  associated  with  the  old  document  are 
likely  to  be  repeated.  If,  on  the  other  hand,  the  author 
chooses  t o  investigate  the  Air  Force  0-2,  all  types  of 
direct  ions  are  ava i  1  ah  1 e  on  how  to  produce  readable 
publications.  Unfortunately,  readability  guidance  is  not 
easily  located,  nor  is  there  any  evidence  for  the  validity 


of  the  recommendations  given.  Therefore,  the  principal 
focus  of  this  paper  is  to  examine  the  current  Air  Force 
procedure  for  incorporat  ing'  the  subject  of  readability  into 
its  documents  and  to  provide  a  useful  synthesis  of  research 
on  the  impact  of  readability  and  its  relationship  to  Air 
Force  publications. 

Me t  ho do  1  pry 

The  primary  purpose  of  this  research  is  to  provide  a 
useful  tool  for  the  military  member  who  is  concerned  uith 
the  subject  of  readability  and  its  relationship  to  military 
publications.  To  accomplish  this  task,  this  paper  begins 
with  an  extensive  review  of  the  topic.  This  review  focuses 
on  two  areas:  (1)  the  significant  readability  formulas  that 
have  developed  over  the  years,  and  (2)  the  articles  which 
deal  with  the  topic  of  readability  of  the  military 
pub  1 i ca  t  ion. 

To  acquire  information  on  significant  readability 
formulas,  the  resources  at  the  Air  Force  Institute  of 
Technology  (AFIT),  the  University  of  Dayton,  and  Wright 
State  University  were  used  to  provide  the  fundamental  data 
base.  However,  to  obtain  relevant  information  on  the 
relationship  between  the  subject  of  readability  and  military 
pub  1  i ca t  ions  it  was  necessary  to  go  beyond  the  resources 
available  to  a  normal  college  library.  In  fact,  a 
computerized  literature  search  using  the  Defense  Technical 
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[nformat ion  Center  (DTIC)  system  proved  to  be  the  primary 
source  of  data  in  acquiring"  articles  which  dealt  with  this 
topic.  The  DTIC  system  consists  of  three  separate  data 
bases,  each  of  which  specializes  in  a  particular  type  of 
information:  (1)  The  Research  and  Technology  Work  Unit 

Information  System  (WUIS)  is  a  database  of  on-going 
Department  Of  Defense  research  and  technology  efforts  at  the 
work  unit  level.  (2)  The  Technical  Report  (TR)  database  is 
a  repository  of  bibliographic  records  of  technical  reports 
submitted  to  DTIC.  And  (3)  the  Independent  Research  and 
Development  (IR&P)  database  is  a  database  of  contractors 
independent  research  and  development  efforts  .shared  with  the 
Department  of  Defense. 

Information  on  the  subject  of  readability  and  on  the 
current  procedures,  techniques,  and  rules  pertaining  to  this 
topic  and  its  relationship  to  United  States  Air  Force 
writing  were  acquired  by  consulting  the  Air  Force’s  0-2. 

All  regulations  related  to  the  subject  of  readability  were 
explored.  As  a  result  of  this  initial  investigation,  the 
following  list  of  regulations  was  chosen  for  more  detailed 
ana  lysis: 

AFR  5-1,  "Air  Force  Publication  Management  Program" 

AFR  5-8,  "Preparing  Air  Force  Publications" 

AFR  8-2,  "Air  Force  Technical  Order  System" 

AFR  10-1,  "Preparing  Written  Communications" 

AFR  35-8,  "Air  Force  Personnel  Testing  System" 
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Additional  official  information  on  this  subject  was  acquired 
by  analyzing'  military  specifications  MIL-M- 38734B  (  M  i  1  i  t  a  r  y 
Specification  Manuals,  Technical:  General  Style  and  Format 

Requ i remen  t  s )  ,  DOD-STD  1685  ( Comprehens i b i 1  i ty  S  tandards 
for  Technical  Manuals),  and  MIL- STD- 1752  (Military  Standard 
Reading  Level  Requirements  for  Prepara1  ion  of  Technical 
0  r  cl  e  r  s  i  Notice  1  . 

The  \:r  Force  unofficial  guide  to  effective 
communication,  Air  Force  Pamphlet  13-2,  Tongue  and  Qu i 1 1 , 
was  also  reviewed  for  its  applicability  to  this  topic. 

Limitations  of  Study 

This  thesis  is  not  intended  to  assess  the  current 
readability  of  regulations,  technical  orders,  or  other  Air 
Force  publications;  nor  is  it  intended  to  act  as  an  advocate 
for  any  particular  readability  formula.  Additionally,  the 
only  readability  formulas  discussed  in  detail  in  this  paper 
are  those  that  have  advanced  the  scope  of  this  field  or 
those  that  directly  affected  the  relationship  of  readability 
assessment  methodologies  and  military  publications. 

Presentat  ion  of  the  Material 

Chapter  two  of  this  thesis  discusses  early  research  in 
readability,  including  an  analysis  of  significant 
readability  formulas.  This  material  is  included  to  provide 
a  fundamental  understanding  of  this  subject,  and  to 
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establish  the  necessary  background  for  comprehending'  aspect, 
of  readability  as  they  pertain  to  the  military  publications. 

Chapter  three  also  examines  readability  formulas. 

These  formulas,  however,  are  significantly  different  from 
those  in  chapter  two.  These  formulas  have  been  used  to 
assess  the  readability  of  military  documents,  or  they  have 
been  newly  developed  and  thus  address  the  topic  of 
readability  from  a  novel  perspective. 

Chapter  four,  which  reviews  the  above  mentioned 
relationship,  is  divided  into  three  sub-sections :  section 

one  contains  an  exploration  of  readability  formulas 
developed  exclusively  for  military  use;  section  two 
investigates  articles  dealing  with  the  relationship  between 
military  publications  and  the  topic  of  readability;  and 
finally,  section  three  analyzes  articles  which  closely 
parallel  the  subject  of  readability. 

Chapter  five  consists  of  an  analysis  of  the  Air  Force’s 
current  procedure  for  ensuring  the  readability  of  its 
publications.  This  analysis  includes:  summaries  of  the 

techniques  that  the  Air  Force  uses  to  obtain  readable 
publications,  a  rationale  (when  available)  for  using  the 
technique,  and  finally  a  critical  assessment  of  that 
ra  t  i ona 1 e . 

Chapter  six  consists  of  conclusions  based  on  the  review 
of  the  literature  and  recommendations  for  future  research. 
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2.  The  Early  Development  of  Readability  Formulas:  1023-1942 

The  principal  focus  of  this  chapter  is  to  provide  the 
novice  with  a  brief  background  on  the  topic  of  readability 
and  a  description  of  significant,  early  readability 
formulas.  This  chapter  describes  the  deve 1 opmen t  of  the 
readability  formula  from  its  birth  in  1023  to  its  maturation 
in  10-12. 

Historical  Background 

From  the  time  when  words  were  first  invented  and  used 
for  communication,  it  has  been  important  for  writers  to 
consider  their  intended  readers.  For  it  is  the  author’s 
choice  of  appropriate  words  and  phrases  which  ot  ten  make  the 
difference  between  the  reader  experiencing  effective 
communication  or  utter  confusion.  From  biblical  times  the 
importance  of  readability  has  been  repeatedly  stressed  in 
the  literature.  One  of  the  favorite  quotations  of  early 
advocates  for  clear  expression  comes  from  1  Corinthians 
14:9:  "Except  ye  utter  by  the  tongue  words  easily  to  be 
understood,  how  shall  it  be  known  what  is  spoken?  For  j  e 
shall  speak  into  the  air." 

T'ne  idea  of  trying  to  measure  readability  probably  had 
its  origins  in  an  1852  article  written  by  Herbert  Spencer, 
entitled  "The  Philosophy  of  Style."  In  this  article  Spencer 
showed  how  the  judicious  use  of  word  choice  by  authors 
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a! loved  the  reader  to  be' ter  comprehend  a  ^iven  subject. 
Spencer  argued  that  humans  have  a  limited  capacity  to 
comprehend  information  so  that  "time  and  effort  devoted  to 
the  decipherment  of  meaning  would  be  subtracted  from  the 
energy  left  to  consider  the  meaning  itself”  (Hirseh, 
1977:73).  Spencer  further  stated  that  efficient  writing  is 
the  key  to  making  articles  m.jim  readable,  and  that  proper 
word  choice  was  the  key  to  that  efficiency. 

In  1389  and  1308  respectively,  N.  A.  Ruberkin  and  F.  W 
kaeding  conducted  separate  studies  which  advanced  Spencer ' s 
qualitative  statements  on  readability.  They  attempted  1 1  > 
provide  a  scientific  basis  for  relating  vocabulary  choice  * 
reading  difficulties  through  word  count  procedures  (Forge, 
1944:543-552). 

The  early  works  on  word  counts  set  the  stage  for  vha ■ 
was  easily  the  most  important  occurrence  of  the 
prereadability  formula  period,  the  publication  in  1921  of 
The  Teacher’s  Word  Book,  a  text  by  E.  L.  Thorndike  (Klare, 
1963:30).  In  this  book,  and  its  subsequent  revisions  in 
1932  and  1944,  Thorndike  examined  samples  of  text  and  then 
listed  words,  based  on  the  frequency  with  which  they 
appeared  in  print  .  The  idea  was  that  the  more  often  a  word 
appeared  in  print,  the  more  easily  that  word  was  thought  to 
be  unde r s t  ood . 

According  to  Klare,  Thorndike’s  research  was  import  ant 
not  only  because  it  provided  the  impetus  for  the  work  of 
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Lively  and  Presser,  who  in  1923  developed  ihe  first 
readability  formula,  but  also  because  it  served  as  a  basis 
for  many  future  readability  studies  (Klare,  1963:30). 

Word  frequency,  however,  was  not  the  only  readability 
variable  to  be  investigated  during'  this  time  period 
(Lamoreaux,  1985:14).  Other  researchers  looked  at  such 
diverse  items  as  sentence  length  (Sherman,  1893:256-268)  and 
the  use  of  multisyllabic  words  (Bear,  1927)  in  their  efforts 
to  develop  readability  formulas,  and  in  turn  to  assess  an 
article’s  readability. 

The  specifics  of  how  some  of  the  early,  significant 
readability  formulas  developed  are  listed  below. 

L i v e 1 y  a nd  Pr ess ly 

The  work  of  Lively  and  Press ly  is  important  to  the 
subject  of  readability  not  only  because  it  represented  the 
earliest  acknowledged  quantitative  study  using'  a  readability 
formula,  but  also  because  it  inspired  future  researchers  to 
develop  their  own  readability  formulas  ( Cha 1 1 ,  1958). 

The  Lively  and  Pressly  technique  (1923)  attempted  to 
determine  the  vocabulary  difficulty  of  a  book.  To 
accomplish  this  task,  they  systematically  sampled  1000  words 
throughout  a  text  and  then  applied  the  following  procedure 
to  those  words:  they  calculated  the  "vocabulary  range"  or 

the  number  of  distinct  words  appearing  in  the  sample;  they 
t  hen  assigned  a  difficulty  index  to  each  word  based  upon 


Thorndike’s  Teacher's  Word  Book.  A  word  found  appearing  in 
Thorndike’s  most  frequent  10Q0  words  vas  given  a  point  value 
of  ten,  the  other  words  in  the  book  were  assigned  point 
values  such  that  the  less  frequent  they  were,  (based  upon 
the  Thorndike  index)  the  lower  point  value  they  would 
receive.  Words  that  did  not  appear  at  all  in  Thorndike’s 
most  common  10,000  words  were  assig’ned  a  point  value  of  zero 
and  were  counted  twice  when  calculating  the  "weighed  median 
index  number."  The  final  step  of  the  Lively  and  Press  ly 
procedure  consisted  of  simply  calculating  the  median  index 
for  the  sampled  words.  The  lover  the  median  index  was 
scored,  the  more  difficult  the  vocabulary  vas  thought  to  be 
( klare ,  1963 : 37-38 ) . 

In  an  effort  to  confirm  the  reliability  of  this  newly 
developed  procedure,  Lively  and  Press  ly  tested  their 
readability  formula  against  sixteen  separate  pieces  of 
reading  material  which  had  been  previously  judged  to  be  of 
different  reading’  levels.  The  results  of  these  tests 
indicated  that  their  computed  weighted  med  an  index  agreed 
consistent  ly  with  the  previously  jvidged  materials’ 
difficult  y  1  eve  1  .  Con  t  i  nu  i  ng’  with  f  he  i  r  initial 
investigation,  Lively  and  Pr^ssly  eventually  established  a 
weighted  median  index  number  for  reading  grade  levels  with,  a 
range  of  difficulty  from  second  grade  through  college 
(Klare,  1963:37,38,75). 
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Lively  and  Press ly’s  work  is  regarded  as  important 
because  it  is  given  credit  for  creating'  an  explosion  of 
interest  in  developing  readability  formulas,  an  example  of 
which  is  the  work  of  Washburne  and  Vogel. 

Kashburne  and  Vogel 

In  1928  Car  1 et  on  Washburne  and  Mabel  Voge 1  applied  the 
Lively  and  Press  !  y  formula  to  TOO  children’s  books.  The;, 
then  determined  the  grade  level  of  the  children  who  liked 
the  books  by  using  the  child’s  score  on  the  Stanford 
Achievement  Test.  This  grade  level  was  then  compared  with 
the  medium  index  score  in  the  Lively  and  Pressly  formula. 
From  this  comparison  a  correlation  coefficient  of  .80  was 
obtained. 

The  Washburne  and  Vogel  findings  were  significant,  not 
only  because  they  provided  the  first  validation  of  a 
readability  formula  using  an  independent,  outside  criterion, 
but  also  because  they  provided  a  starting  point  from  which 
these  authors  could  develop  their  own  readability  formula 
(Klare,  1903:39). 

The  Vogel  and  Washburne  procedure  is  of  special 
interest  to  the  student  of  readability  because  it  is 
considered  to  be  the  precursor  of  the  modern  readabi ! ity 
formula.  Their  innovative  technique  consisted  of:  choosing 

a  sample  passage  of  1000  words  from  the  book  to  be  analyzed, 
counting  the  number  of  different  words  in  that  sample, 
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count  ;  ng  the  total  number  of  prepos  i  t  ions  in  'he  sample, 
determining'  the  number  of  words  not  listed  in  Thorndike’s 
list  of  10,000  vords,  and  then  counting  the  number  of  simple 
sentences  vhich  appeared  in  a  sample  of  75  sentences.  Once 
these  steps  vere  ac comp  1  i shed ,  the  regression  equation  vus 
ust'd  to  determine  the  reading  score  (Klare,  190.3:39).  I'h  i 
score  was  computed  by  using  the  following  formula: 

A  .085  B  +  .101  C  -  .004  0  -  .111  E  r  17.1.3 

where  A  was  equal  to  the  reading’  score,  B  was  equal  t  ,  the 
number  of  d  i  f  f  eren  *  words  in  the  sample,  C  was  equal  t  o 
’ota 1  number  of  prepos i t ions ,  D  was  equal  f  o  ' he  numb<  r 
words  not  listed  in  Thorndike’s  10,000  words,  and  H  .was 
equal  to  the  number  of  simple  sentences  in  a  tested  samp !  * 
of  75  sentences. 

P>  1  c  h 

Another  author  who  explored  t  lie  subject  of  roadabi  1  i  r :. 
during  this  time  frame  (1928)  was  Edward  Dolch.  Dolch 
examined  text  books  written  for  grades  one  through  four.  [n 
his  analysis  of  articles’  readability  Dolch  used  the 
following  five  indices  of  difficulty:  the  percentage  >■:' 

difficult  words  (based  on  the  Dolch’s  Combined  Word  Studx 
List ,  the  percentage  of  different  words  (again  using  the 
Dolch  combined  word  study  list),  the  degree  of  difficult).  ■  f 
words,  the  med i an  frequency  of  difficult  words,  and  finally 
the  degree  of  difficulty  fur  any  supplementary  reading 
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materia!  (Du!  eh,  1928:170-189).  Dolch  found  a  aide  degree 
of  difference  in  the  difficulty  level  of  texts  supposedly 
written  for  the  same  grade  level. 

Lev  e  r  enz 

A.  S.  Lewerenz  explored  the  subject  of  readability  from 
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bizarre.  Rather  than  count  ing  words  which  began  wi’h 
various  letters  of  the  alphabet,  Lewerenz  evaluated  art  icles 
based  on  t  he  ratio  of  Anglo-Saxon  words  to  the  ratio  of 
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,,or  Jb  si  t  h  Grt'i-k  or  Roman  t  La  t  in)  tier  i  vat  ion.  From  f  he 
ra  t  i  os  obtained  he  assigned  various  grade  levels  to  tin- 
articles  (Klare,  1963:41).  The  more  Anglo-Saxon  words 
present  in  the  article,  the  easier  it  was  considered 
r  e  a  d  . 

.1  ohn  s  on 
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determining  what  percentage  of  the  total  words  examine.;  w .  r 
polysyllable  words.  Johnson  claimed  that  In  using  this 
method,  a  hook’s  reading  difficulty  level  could  :a- 
determined  in  approximately  30  minutes. 

Johnson’s  research  cited  specific  evidence  for  the 
reliability  of  using’  the  polysyllabic  word  count  as  a 
determinate  of  reading  level.  This  informat  ion,  which  he 
claimed  subs  L  an  t  i a  U  J  h i s  conclusions,  was  ob'ained  f r -m 
four  different  sources:  studies  which  show  better 

understanding  by  students  who  read  a  story  in  men-  s\  1  l.d  i 
words  than  by  those  who  read  it  in  polysyl  1  ah i c  words,  in 
inverse  re  1  at  ion  which  exists  between  poly-yl  labir  r«!s  an 
Thorndike’s  word  f  rcijuency  index, 


:  t; 


the  increased  use  of 


polysyllabic  words  in  texts  designed  to  be  used  by  higher 


grade  levels,  and  the  close  relationship  between  the  number 
of  polysyllabic  words  and  the  number  of  technical  words  used 
in  the  seven  books  he  examined  (Johnson,  G.,  1 930 : 283- 2b7 ) . 

A  few  years  later  Thorndike  examined  the  subject  of 
readability  from  a  slightly  different  per  spec t  i  ve  . 


The  rnd i ke 

In  1934  Thorndike  developed  a  procedure  for  determining 
the  readability  of  a  book.  His  methodology  consisted  of  '  h- 
following'  two  steps:  sample  at  least  10,000  words  from  the 

text,  and  count  the  number  of  words  included  in  each  of  the 
different  categories  of  the  Teacher’s  Word  Book.  Based  on 
this  count  it  was  then  possible  to  determine  the  grade  level 
at  which  the  book  was  written. 

It  is  interesting  to  note  that  Thorndike’s  readability 
formula  perhaps  best  characterized  the  early  work  in 
readability.  This  period  was  shaped  by  an  emphasis  on 
vocabulary  as  the  primary  determinate  to  predicting  a 
subject's  difficulty  level,  a  dependence  on  the  T e a c h e r ’ s 
Word  Book  as  a  method  of  determining  a  vocabulary’s 
difficulty,  and  the  use  of  relatively  simple  mathematical 
formulas  for  calculating’  a  work’s  readability  (Klare, 
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Oj emanu 

In  1934  Ralph  Ojemann  approached  the  subject  of 
readability  from  a  novel  perspective.  He  sought  to 
empirically  determine  which  factors  were  most  closely 
related  to  reading'  difficulty.  The  factors  Ojemann  chose  to 
investigate  fell  into  three  broad  areas.  The  first  area, 
sentence  factors,  »as  subdivided  into:  the  number  of  simple 

sentences,  the  number  of  complex  sentences,  the  number  of 
compound  sentences,  the  number  of  dependent  clauses,  the 
number  of  preposit ions,  the  number  of  prepositions  plus 
infinitives,  the  average  leng'th  of  the  dependent  clauses, 
and  the  ratio  of  total  words  in  the  independent  clauses  to 
the  total  words  in  a  section. 

The  second  area  consisted  of  vocabulary  factors.  This 
section  included  six  subsections:  the  percentage  of  words 

in  Thorndike’s  first  1000  words,  the  percentage  of  words  in 
Thorndike’s  first  2000  words,  the  percentag'e  of  words  known 
by  70  percent  of  sixth  grade  pupils,  the  percentage  of  words 
known  by  90  percent  of  sixth  grade  pupils,  the  mean 
difficulty  of  different  words,  and  the  mean  difficulty  for 
each  word. 

The  final  area  Ojemann  sought  to  investigate  was 
qualitative  factors.  Included  in  this  category  »ere:  the 

concreteness,  as  opposed  to  the  abstraction,  of  a  relat  ion, 
the  obscurity  of  expression,  and  the  incoherence  of 


expression  (Klare,  1963:44-45). 


As  a  result  of  his  extensive  investigation  Oje.naun 
found  that  there  vas  a  correlation  at  the  .60  level  or 
better  between  his  vocabulary  factors  and  an  article’s 
reading  difficulty.  For  the  sentence  factors,  only  the 
number  of  simple  sentences,  the  number  of  propositions,  and 
the  number  of  prepositions  plus  infinitives  showed 
c or  re  1  a  t  !  or.  t  o  t  ha  t  1  eve  1  .  Add  i  t  i  ona  1  1  y  0  j emanr.  o  on c  i  r.ded 
‘hat  the  j'-ia  i  t  at  i  ve  factors  were  important  to  an  at"  Ice’s 
readability,  although,  he  could  not  compute  a  mathematical 
- . r r e 1  a t 1  on  for  them  (Ojemann,  1934:1  1  -32). 

Dale  and  Tyler 

Dale  and  Tyler's  study  (1  934)  was  similar  to  the  vor/. 

■  f  Ojemann  in  both  content  and  form. 

Th.es e  authors  sought  to  develop  a  means  by  which  they 
>i.  1  d  determine  the  factors  influencing  the  reading’ 
difficulty  for  adults  of  limited  reading  ability.  In  their 
study  they  looked  at  25  different  items  to  see  which  had  a 
strong  correlation  with  reading  difficulty.  The  items  they 
investigated  could  be  roughly  grouped  into  the  following 
eight  categories:  technical  words,  easy  words  (which,  they 

defined  words  common  to  both  Thorndike's  first  1000  word 
and  to  the  list  of  the  International  Kindergarten  I'nion), 
difficult  nontechnical  words,  the  type  and  length  of  the 
sentences  used  (such  as  compound,  complex,  or  simple),  the 
number  of  clauses  or  prepositional  phrases,  the  number  of 


persona!  pronouns,  the  number  of  monosyllabic  words,  and 


other  factors  (Klare,  1963:46-47).  As  a  result  of  their 
investigation  Dale  and  Tyler  developed  the  following 
regression  equation  which  yielded  a  correlation  of  .51  with 
the  c  r i ter ia: 

A  =  -9.4  B  -  0.4  C  +  2.2  D  +  114.4  { - / -  9 . 0  ) 

where  A  was  defined  as  the  percentage  of  adults  of  third  to 
fifth  grade  reading  ability  who  could  understand  a  passage, 
B  was  equal  to  the  number  of  different  technical  words  in 
the  passage,  C  equated  to  the  number  of  hard  non- technical 
words,  and  D  was  the  number  of  indeterminate  clauses  presen 
(Dale  and  Tyler,  1934:384-412). 


McC 1  us  ky 

McC'lusky  was  the  next  researcher  to  expand  the  scope  o 
readability.  In  his  1934  studies  he  used  speed  of  reading 
tests,  rather  than  comprehension,  as  an  index  of  a  book’s 
readability.  McClusky  found  that  "easy"  material  contained 
short  sentences  and  simple  terminology,  while  "hard" 
material  contained  complex  sentences  and  technical  or 
unfamiliar  words  (Klare,  1963:206). 

Although  unique,  McClusky’ s  speed  of  reading  tests  did 
not  appear  to  have  much  influence  on  future  readability 
studies.  The  next  researchers  who  had  did  have  such  an 
impact  were  Gray  and  Leary. 


Gr.'ty  ami  l.carv 


In  1935  Grii>  and  Leary  published  a  classic  work  on  the 
subject  of  readability  entitled  What  Makes  a  Book  Readable. 
This  text  was,  at  that  time,  the  most  thorough  work  on  the 
subject  ever  printed.  As  a  starting  point  for  researching 
this  book,  and  in  turn  for  developing  their  own  readability 
formula,  Gray  and  Leary  investigated  all  previous  work  on 
the  subject.  They  consulted  with  approximately  100  experts 
in  the  field  as  well  as  over  170  library  patrons.  From  this 
initial  investigation  Gray  and  Leary  developed  a  list  of  ~>-'J 
factors  which  could  influence  an  article’s  readability.  The 
factors  were  then  classified  into  four  broad  categories: 
context,  style  of  expression  and  presentation,  format,  and 
general  organization.  Of  these,  the  style  section  was 
chosen  for  further  investigation. 

As  a  result  of  running  numerous  statistical  tests  to 
see  which  of  the  factors  best  correlated  with  reading  score, 
the  researchers  developed  the  following  formula: 

A  -  - .91029  B  +  . 099G 1 2  C  -  .02094  D  -  .03313  E 

-  .01485  F  +  3.774 

where  A  is  equal  to  the  average  comprehensive  score  for 
adults  of  limited  reading  ability,  B  is  equal  to  the  number 
of  hard  words,  C  is  the  average  number  of  personal  pronouns, 
D  is  the  average  number  of  words  per  sentence,  E  is  the 
percentage  of  different  words,  and  F  is  the  number  of 
prepositional  phrases.  Subsequent  research  found  that  this 
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formula  correlated  at  the  .6435  factor  with  the  criteria 
(Klare,  1963:49). 

The  last  of  the  readability  formulas  to  appear  in  the 
1930s  was  the  revised  Washburne  and  Vogel  formula. 

Washburne  and  Morphett 

In  1938  Carleton  Washburne  and  Mabel  Vogel  Morphett 
amended  their  original  readability  formula.  These  changes 
were  designed  with  the  following  purposes  in  mind:  to 

simplify  the  computations  involved  in  computing’  grade  level 
(without  changing  the  accuracy  of  the  original  formula),  as 
well  as  to  compensate  for  the  skewing  at  the  extremes  of  th 
grade  distributions,  which  were  major  problems  for  the 
original  formula. 

The  new  procedure  they  developed  included  the  followir. 
steps:  select  a  sample  of  1000  words,  count  the  number  of 

different  words,  count  the  number  of  uncommon  words  (those 
not  found  in  Thorndike’s  first  15  00),  and  filially  count  the 
number  of  simple  sentences  in  75  sample  sentences.  The 
results  obtained  were  then  applied  to  the  following  formula 

A  --  .00255  B  +  .0458  C  -  .0307  D  +  1.294 
where  A  is  equal  to  grade  placement ,  Bis  equal  to  the 
number  of  different  words,  C'  is  equal  to  the  number  of 
different  uncommon  words,  and  D  is  the  number  of  simple 


sentences  (Washburne  and  Morphett,  1938:355-364). 


In  1930  Irving'  Lorg'e  published  the  next  significant 
work  on  the  topic  of  readability.  This  work,  inspired  in 
part  by  the  Gray  and  Leary  formula,  is  considered  important 
in  this  field  because  its  formula  was  the  first  readability 
measurement  to  be  used  extensively  in  occupations  other  than 
educ  a t  i on . 

Although  the  Lorge  formula  was  originally  designed  to 
serve  as  "an  estimate  of  the  reading  grade  at  which  the 
average  school  child  will  be  able  to  answer  about  55  percent 
of  the  questions  concerning'  detail,  appreciation,  import  , 
vocabulary,  and  concept  with  adequate  completeness  and 
correctness,"  this  formula  has  never- the- 1  ess  seen  wide¬ 
spread  use  (Lorg'e,  1959:1). 

The  attractiveness  of  this  formula  can  be  at’ribut  -d  to 
tile  fact  that  it  required  only  three  elements  to  compute: 
the  average  sentence  length,  tne  number  of  prepositional 
phrases,  and  the  number  of  hard  words  (based  on  the  Dale 
list).  Unfortunately,  the  original  1939  Lorge  formula  was 
found  to  contain  errors,  and  it  was  necessary  for  Lorge  to 
recompute  his  formula.  This  recomputation  was  published  in 
19-13.  The  revised  formula  is  calculated  as  follows: 

A  =  .  0  G  B  +  .10  C  +  .10  D  +  1.9  9 

where  A  is  the  grade  placement  level,  B  is  the  average 
sentence  length  in  words,  C  is  the  number  of  prepositional 


phrases  per  100  vorcis  ,  and  D  is  the  number  of  different  hard 
words  per  100  words  not  on  the  Dale  list. 

Kess 1 er 

Like  Lorg'e’s  system,  Edward  Kessler’s  readability 
measurement  technique  was  inspired  by  the  work  of  Gray  and 
Leary.  The  Kessler  formula,  however,  was  a  shortened 
version  of  the  Gray  and  Leary  formula.  His  methodology 
consisted  of:  selecting'  ten  paragraphs  of  approximately  100 

..oids  each,  counting'  the  number  of  words,  sentences,  and 
different  hard  words  in  the  passages ,  then  calculating  the 
average  sentence  length  in  number  of  words  and  the  number  "f 
different  hard  words  per  100  words,  and  finally  comparing 
those  calculated  averages  with  the  standards  developed  by 
Gray  and  Leary  ( K 1  are ,  1  903  :  55  )  . 

Kessler’s  work  was  the  last  formula  to  be  developed 
during  the  early  readability  period,  a  period  characterized 
by  experimentation  in  defining  what  makes  a  piece  of  text 
readable.  Kessler's  work  was  also  the  last  readability 
formula  developed  before  the  arrival  of  Rudolf  Flesch,  a 
researcher  who  revolutionized  the  field  of  readability. 


3.  The  Recent  Development  of  Readability  Formulas:  10  13-  1 0 8 ' 

The  recent  development  of  the  readability  formula 
starts  in  1943,  with  the  arrival  of  Rudolf  Flesch  and  his 
Flesch  formula.  With  this  single  event  the  field  of 
readability  was  dramatically  changed.  The  Flesch  formula 
almost  overnight  became  the  standard  by  which  most 
subsequent  readability  formulas  were  either  developed  or 
compared.  It  also  served  as  the  source  formula  for  many 
future  examinations  on  the  topic  of  readability  and  its 
relationship  to  the  military  publication  (the  subject  of 
chap  ter  4 )  . 

In  this  chapter,  the  Flesch  formula  and  the  subsequent 
research  it  inspired  are  examined.  Additionally  recent 
research  extending  the  topic  of  readability  are  commented 
on  . 


Flesch 

Probably  the  best  known  and  most  referenced  researcher 
in  the  field  of  readability  is  Rudolf  Flesch.  Flesch’s 
formulas  on  this  topic  are  widely  used  in  assessing  the 
readability  of  a  variety  of  texts.  The  reasons  for  this 
extensive  use  are  many,  but  include  the  formula’s  basic 
simplicity  and  the  fact  that  Flesch  was  a  master  at 
promoting  it.  In  a  series  of  books  and  articles  Flesch 
advertised  his  formula,  his  views  on  the  importance  of 
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readable  writing",  and  his  concept  of  readability  to  vr  iters 
in  journalism,  business,  government,  and  the  military 
services  (  K  1  a  r  e  ,  1963:56). 

Flesch’s  first  formula  on  readability  (1943)  vas 
revised  several  times.  The  first  revision  vas  needed  because 
Flesch’s  formula  had  been  based  on  errors  present  in  the 
original  Lor g e  data,  data  which  vas  absolutely  essen?  ia!  f'.<r 
the  accuracy  of  Flesh's  computations.  The  second  revision 
vas  required  because  many  people  thought  that  the 
computation  of  Flesch’s  formula  vas  too  difficult. 

In  response  to  these  criticisms,  Flesch  designed  -a 
revised  estimate  of  readability  (1948)  which  consisted  of 
two  different  factors:  reading  ease  (RE)  and  human  interest 

(MI).  To  measure  the  factor  of  reading  ease,  Flesch 

% 

examined  randomly  selected  sample  passages,  measured  the 
number  of  syllables  (SY)  per  100  words,  and  then  measured 
the  average  number  of  words  per  sentence  (ST).  He  then 
placed  the  resulting  numbers  in  this  formula: 

RE  =  206.835  -  .846  (SY)  -  1.015  (ST) 

The  RE  value  obtained  vas  then  correlated  with  grade  level. 
If  the  RE  score  was  computed  to  be  between  90-100,  the 
material  was  considered  to  be  equal  to  fourth  grade  level 
material.  A  passage  which  scored  between  80-90  vas  equa 1  to 
fifth  grade  level,  and  a  score  of  between  0-30  was  written 
to  the  college  level  (Flesch,  1974:247). 


The  human  interest  score  Fleseh  developed  indicates 
the  liveliness  of  a  writing'  style.  This  score  is  computed 
by  first  counting'  the  number  of  "personal  words"  ( PW )  per 
100  words  and  then  the  number  of  personal  sentences  (PS)  per 
1000  sentences.  The  resulting  numbers  are  then  applied  to 
the  following  formula: 

HI  --  3.6  35  PW  +  .314  PS 

If  the  HI  score  was  between  0  to  10,  the  writing  style  w  as 
considered  to  be  uninteresting'.  If  the  score  was  from  10  to 
20,  the  style  was  mildly  interesting,  and  if  an  article 
scored  60  or  more  it  was  said  to  be  written  with  a  dramatic 
writing’  style  (Fleseh,  1916:195). 

Da  1 e  and  Cha 1 1 

The  second  most  referenced  formula  in  the  field  a f 
readability  can  be  attributed  to  the  work  of  Edgar  Dali'  and 
Jeanne  Cha 1 1 .  Their  procedure,  which  was  designed  to 
correct  certain  inadequacies  of  the  original  Fleseh  formula, 
first  appeared  in  1948. 

Dale  and  Cha 1 1 ,  in  structuring  their  formula,  thought 


*1  h  cl  t  ct 

larger  word  list  would 

be  a 

better  predictor 

o  f 

art  i  c  1  e 

’  s  reading  difficulty 

level 

(particularly  at 

the 

upper  levels  of  difficulty)  than  the  Dale  769-word  list  . 

They  believed  that  the  count  of  personal  references  as  used 
by  Fleseh  was  an  unnecessary  and  time  consuming  step,  and 
that  a  more  efficient  readability  formula  could  be  developed 
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us  i  tig-  only  a  word  factor  and  a  sentence  far  t  or 


a  s 


predictors  (Klare,  1963:59-61). 

The  technique  these  researchers  used  for  comput  in-' 
their  readability  formula  consisted  of:  se1 °ct ing  sample 

passages  of  the  material  to  be  analyzed  that  contained  at 
least  100  words,  computing  the  average  sentence  1  eng:  h  in 
words  for  those  passages,  computing  the  percentage  of  words 
not  included  in  the  Dale  list  of  3000  voids,  and  then 
applying  the  obtained  results  to  the  following  formula: 

A  -  .1579  B  +  .0496  C  +  3.635 

where  A  is  the  reading  grade  score  of  a  s  t  uden  ?  who  could 
answer  50  percent  of  the  test  questions  on  a  passage 
correctly,  B  is  the  percentage  ot  words  not  present  in  the 
Pcile  list  of  3000  words,  and  C  is  the  average  sentence 
length  in  words  (Dale  and  Chall,  1943:11-20). 


Karr,  Jenkins,  and  Pa  iter  sun 

Farr,  Jenkins,  and  Patterson  also  developed  a 
readability  formula  (1951)  which  was  a  simplified  version  of 
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tile  part  of  the  user  would  not  be  needed  (Klare,  1963:64). 


T"  use  their  formula  all  that  vas  necessary  was  >n 
accomplish  the  following  three  steps:  select  a  100-vord 

sample  from  the  material  to  be  analyzed,  count  the  number  of 
one  syllable  words  per  100  words,  and  calculate  the  average 
sentence  length.  Once  these  steps  were  accomplished  the 
result  ing  numbers  were  applied  t o  the  following  formula: 

A  -  1.509  B  -  1.015  C'  -  31.57 


where 

A  was  equal 

t  o  t  he  " 

new  read 

ing  ease  index,"  B  was 

equa  1 

to  the  numbe 

r  of  one 

syl  1  ab  1 

e  words  per  100  words, 

C  w  a  s 

the  average 

sentence 

1  engt  h 

(Klare,  1963:64). 

Gunn  i  tig 

Similar  to  Dale  and  C'ha  1  1  ,  as  well  as  Farr,  Jenkins, 
and  Paterson,  Robert  Gunning  also  revised  the  Flesch  Reading 
Ease  formula  (1952).  He  accomplished  this  revision  by 
replacing  the  total  syllable  count  with  a  count  of  words 
having  three  or  more  syllables.  The  procedure  for  using 
this  formula  consisted  of  determining  the  average  sentence 
length  ( in  number  of  words),  counting  the  number  of  words  of 
three  or  more  syl tables  to  obtain  the  percentage  of  hard 
words,  and  then  adding  those  two  factors  together.  The 
resulting  sum  from  this  addition  was  multiplied  by  .4  to 
obtain  the  Fog  Index  (or  the  reading  grade  level  required 
for  an  accurate  understanding  of  the  material)  (Klare, 
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1963:65) . 


The  Fog'  Index,  according'  to  Klare,  was  presumably 
designed  to  be  applicable  for  reading  material  that  ranged 
in  difficulty  level  from  sixth  to  twelfth  grade  (Klan-, 
196.1:79).  Additional  evidence  which  supports  this 
conclusion  comes  from  Gunning  himself,  who  stated  that  if 
art  icle  scored  above  a  twelve  on  his  Fog  Index  it  was  in 
danger  of  being  ignored  or  misunderstood  and  should  pr>  ‘bah 
he  rewritten  (Gunning,  1952:33). 

Tav  1  or 

Tn  1951  Wi Ison  Taylor  examined  the  subject  of 
readability  from  a  innovative  perspective.  A  doctoral 
student  in  psychology,  Taylor  approached  the  measurement 
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group  made  when  blank  spaces  were  inserted  in  a  prose  samp 
to  replace  every  fifth  or  sixth  word"  ( H i r s c h ,  1  977:9  5). 

then  correlated  the*  number  of  correct  guesses  with  a  read  i 
grade  level  .  Tn>  I  or  ’  s  technique  of  measuring  read  ah  i  11'.'' 
romp  1 c  t  ing  or  closing  sentences  came  to  he  cal  led  the  "Cl“ 
procedure"  (Taylor,  W.,  1955:161-465). 

Subsequently  it  was  found  that  Taylor’s  Cloze  procedu 
yielded  results  which  correlated  remarkably  well  not  only 
with  t  rad i  t  i ona 1  rcadah i  1  i  t  y  f  ormu 1  a  s  cores ,  bu t  wit  h 


rea  ding  rompn-hens  i  on  s  cores  as  we  1  1  (Taylor,  W.  ,  1957  :  1 9 - 

2  5  )  . 

One  of  the  traditional  readability  formulas  with  which 
the  Close  formula  showed  a  h i gh  correlation  is  the  Automated 
Readability  Index.  This  formula/mechanism,  invented  by  R. 

J .  Smith  and  E.  A.  Senter,  is  discussed  in  detail  below. 

Sm i  t  h  and  Son i e r 

In  1907  Smith  arid  Senter  created  the  Automated 
Readability  Index  (ART).  Th i s  formula/mechanism  was 
designed  to  automate  the  computation  of  a  text  's 
readability.  The  formula  portion  of  the  AR1  has  been 
revised  several  times,  with  the  most,  recent  revision 
occurring  in  1970  (Smith  and  Kincaid,  1970:459).  The 
revised  formula  is  calculated  as  follows: 

RGL  -.50  (words/sentence)  +  4.71  (  s  t  rok.es/*  ord ) 

-  22.43. 

The  mechanical  portion  consists  of  cin  attachment  to  an 
electric  typewriter.  This  device  tabulates  the  number  of 
words  per  sentence  and  the  number  of  strokes  per  word. 

Based  on  this  tabulation  an  RGL  is  then  determined  (Smith 
and  Senter,  1 9 0 7 : ab s t  rae t  )  . 

It  is  interesting  to  note,  that  the  Smith  and  Senter 
formula  marked  the  end  of  an  era,  fur  during  the  latter  lui  1 f 
of  the  1950s  the  widespread  interest  in  readability  formulas 
appeared  to  vane.  However,  despite  this  lessening  of 
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general  interest,  specific  researchers  began  to  examine  the 
applicability  of  using  readability  formulas  to  measure  the 
readability  of  military  publications. 
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4.  Rea dab  1  1  i  t  y  Formulas  and  Their  Applicability  fo 
the  Military  Publication 

This  chapter  examines  the  use  of  readability  formulas 
as  assessment  tools  for  military  documents.  The  material 
covered  is  addressed  from  the  following  perspective:  First, 

the  readability  formulas  developed  exclusively  for  military 
use  v i i 1  be  explored.  Then  research  examining  the 
applicability  of  using'  readability  formulas  with  military 
documents  will  be  summarized.  Finally,  articles  examining 
concepts  paralleling'  the  subject  of  readability  and  its 
relationship  to  the  military  document  vill  be  described. 

Cay  lor,  Sticht,  Fox,  and  Ford 

Taking  a  traditional  approach  to  the  topic  of 
readability  (1973),  Cay  lor,  Sticht,  Fox,  and  Ford  focused 
their  research  on  developing  a  simple,  easy - t o - adm  :  n  i  s ; er 
readability  formula.  This  formula  was  designed  to  predict 
the  reading  grade  level  of  U.S.  Army  job  reading  materials 
in  a  valid  and  reliable  manner  (Cay  lor  and  others,  1973:'.’). 

The  authors  be  1  ieved  that  it  was  necessary  to  develop  a 
nev  readability  formula  which  could  address  the  uniqueness 
of  Army  needs,  which  were  characterized  by  young  adult  male 
readers  with  low  reading’  grade  levels  and  job  reading 
material  containing  heavy  dosages  of  technical  verbage  and 
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jargon . 


As  part  of  the  process  for  developing  their  formulas, 
these  researchers  conducted  an  extensive  literature  review 
on  the  subject  of  readability.  From  that  hunt  15  suitable’ 
variables  were  chosen  for  additional  investigation.  These 
variables  were  grouped  under  the  following  categories:  (1) 

sentences,  (2)  words  per  sentence,  (3)  independent  clauses, 
(3)  words  per  independent  clause,  (o)  one-syllable  words, 

(6)  difficult  words,  (7)  different  difficult  words,  (3) 
different  words,  (9)  three-or-more-sy  1  1 ab 1 e  words,  (10) 
total  number  of  syllables,  (11)  total  number  of  letters, 

(12)  syllables  per  sentence,  (13)  number  of  letters  per 
sentence,  (14)  seven-or-mcre- let  ter  words,  and  (15) 
different  three-or-more-sy 1 lable  words  (Caylor  and  others, 
1973:  7-8).  These  variables  were  tested  using  correlation 
techniques.  As  a  result  of  these  tests  the  following 
preliminary  readability  formula  was  developed. 

RGL  =  20.43  -  (.11)  (A) 

where  A  equals  the  number  of  one-syllable  words  present  in  a 
150- word  passage. 

This  formula  was  eventually  simplified  and  dubbed  the 
FORCAST  formula,  named  from  its  authors  FORd ,  C'Aylor,  and 
STicht.  The  FORCAST  formula  is  calculated  as  follows: 

FORCAST  RGL  -  20  -  A/ 1 0 

where  A  equa I s  the  number  of  one-syllable  words  present  in  a 
150-word  passage  (Caylor  and  others,  1973:15). 


This  formula  was  subsequently  validated  by  examining' 
the  intercorrelat  ions  among"  FORCAST,  Flesch,  and  Bale-Chall 
readability  indices  and  scaled  reading  grade  level  scores 
(C'aylor  and  others,  1973:1  6  ). 

Kincaid,  Fishburne,  Rogers,  and  C'hissom 

Kincaid,  Fishburne,  Rogers,  and  C'hissom  also  sought  "> 
develop  readability  formulas  (1975)  which  would  be 
appropriate  for  military  personnel.  Specifically,  they  used 
the  techniques  developed  by  Cay  lor  and  others  (  1  973)  to 
modify  existing  readability  formulas  for  use  with  the  L' .  S  . 
Navy  technical  material. 

Of  the  numerous  formulas  they  examined,  the  most 
interesting  is  the  recalculated  Flesch  Reading  Ease  formula. 
This  formula  stands  out  because  it  eventually  became  the 
Department  of  Defense  ( DOD )  standard  for  calculating  the 
reading  grade  level  of  its  Technical  Orders.  This  formula, 
which  is  known  as  the  FI esch-Kinca i d  formula,  or 
alternatively  as  the  Kincaid  formula,  is  calculated  as 
f  o 1  1 ows : 

A  =  .39  (B)  +  11.8  (C)  -  15.59 

where  A  equals  grade  level,  B  equals  words  per  sentence,  and 
C'  equals  syllables  per  word  (Kincaid  and  others,  1973:1  4). 

Besides  becom :ng  the  DOD  standard  for  calculating  the 
readability  of  Technical  Orders,  the  Kincaid  formula  was 


also  one  of  the  earliest  readability  formulas  to  be 


computerized.  One  of  the  first  researchers  to  use  these 
newly  developed  formulas  to  determine  the  readability  level 
of  Air  Force  regulations  were  Hooke,  DeLeo,  and  Slaughter. 


Hooke,  DeLeo,  and  Slaughter 

In  1979  Hooke,  DeLeo,  and  Slaughter  attempted  to 
determine  the  readability  of  Air  Force  publications  which 
had  been  rewritten  to  supposedly  match  the  reading  grade 
1  eve  1  (  RGI. )  of  the  intended  users.  The  regulations  chosen 

for  investigation  had  to  be  suitable  not  only  for  FORCAST 
readability  estimation  (see  Cay  lor  1  973),  but  for  the 
development  of  CLOZE  tests  as  well  (Hooke  and  others, 
1979:8).  In  their  search  for  suitable  material  the  authors 
found  seven  regulations  which  met  those  requirements:  Ait- 

Force  Regulations  4-2,  6-1,  25-5,  39-18,  66-39,  144-1,  and 

1 9 0-23. 

After  analyzing  the  readability  of  these  regulations 
and  the  relationship  to  the  intended  users,  the  authors 
determined  that  many  individuals  responsible  for  writing  Air 
Force  regulations  were  not  able  to  use  the  FORCAST  formula 
to  accurately  estimate  the  reading  grade  level  of  the 
publication  they  had  written,  nor  were  they  able  to  achieve 
the  target  reading  grade  level  of  their  intended  audience. 
This  was  especial  1 y  true  when  the  target  RGL  was  below  the 
tenth  grade.  In  occupations  -where  the  Reading  Grade  Levels 
were  relatively  high,  and  there  was  no  negative  literacy 


gap,  comprehension  of  the  re^ulat  ions  vas  at  an  aecej  tab’e 
level;  however,  where  the  RGL  of  the  users  was  low  ('9.5), 
there  did  tend  to  be  a  1  iteracy  gap  and  thus  correspondingly 
inadequate  comprehension  (Hooke  and  others,  1979:15-16). 

Hooke  and  others  recommended  that  writers  of  Air  Force 
regulations  should  receive  additional  training-  in  the  use  of 
the  FORC'AST  formula  and  in  techniques  to  decrease  the  RGL  of 
their  texts.  Additionally  these  researchers  concluded  that 
the  practical  problems  of  simplifying'  Air  Force  material 
below  the  tenth  grade  level  might  be  insurmountable  because 
most  publications  must  contain  a  large  number  of  fixed 
polysyllabic  terms  (Hooke  and  others,  1979:13-17). 

Other  authors  equally  concerned  about  the  problems  of 
writing  Air  Force  material  to  a  particular  Reading  Grade 
Level  are  those  individuals  assigned  to  produce  Procedural 
Manuals  and  Technical  Orders. 

Smith  and  Kincaid 

In  1979  Smith  and  Kincaid  attempted  to  determine  the 
applicability  of  using'  the  Automated  Readability  Index  to 
examine  the  readability  of  Lnited  States  Air  Force  Technical 
Orders.  Specifically,  they  sought  to  assess  the  value  of 
this  i ns t rumen t  as  a  valid  and  reliable  predictor  of  the 
readability  of  the  TOs .  They  also  wished  to  determine  the 
ease  of  use  of  the  formula,  as  well  as  any  limitations  that 
might  be  present. 
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Based  un  their  analysis  the  following  four  conclusions 
were  drawn:  First,  ARI  is  a  reliable  measurement  instrument 

yielding  a  test-retest  correlation  consistently  above  the 
.985  level.  Second,  the  validity  of  the  formula  is  suspect, 
particularly  at  differentiating  easy  from  medium  versions  of 
material.  Third,  ARI  is  easy  to  use  and  it  appears  tc  be 
more  efficient  and  economical  than  traditional  manual 
techniques.  Finally,  if  authors  attempts  to  write  material 
specifically  to  achieve  a  low  readability  score  they  may  in 
fact  degrade  the  readability  of  the  document  (Smith  and 
Kincaid,  1  970:457-484  )  . 

Johnson,  Re  leva,  and  Stafford 

Johnson,  Re  1  ova,  and  Stafford  also  conducted  an 
investigation  (1972)  into  the  relationship  between  the 
readability  of  Air  Force  procedural  manuals  and 
discrepancies  involving  non-compliance  with  those 
procedures.  The  study  was  undertaken  in  response  to  t he 
United  States  Air  Force’s  concerns  over  non-compliance  with 
written  policies  and  procedures.  The  focus  of  the 
invest igat  ion  was  limited  due  to  the  large  number  of 
publications  in  existence.  Rather  than  trying'  to  sample  all 
Air  Force  publications,  these  investigators  chose  to  focus 
their  research  on  administrative  procedures. 

In  examining  the  publications,  these  researchers  had 


two  object  ives  in  mind: 


to  assess  the  readability  gap  which 


existed  between  procedural  manual s/re^ulat ions  and  their 
target  audiences,  and  to  determine  if  a  relationship  existed 
between  the  readability  gap  and  the  extent  of  non-compliance 
with  the  procedures  in  those  publications  (Johnson  and 
o  t  hers ,  19  7  2:1  2). 

The  readability  formula  with  which  they  analyzed  the 
material  was  the  Flesch  formula.  The  reading  grade  ! eve . s 
of  the  respective  audiences  for  the  various  pub li cat ions 
were  determined  by  analyzing  the  data  obtained  from  the 
triform  Military  Record  ( UMR )  .  The  l"MR  served  as  a  source 
document  through  which  Johnson,  Re  1  ova ,  and  Stafford  could 
determine  the  reading  grade  level  for  any  Air  Force 
Specialty  Code  (AFSC). 

As  a  result  of  this  investigation,  Johnson,  Re ! ova ,  and 
Stafford  concluded  that  the  number  of  errors  which  occurred 
when  procedural  tasks  were  performed  were  direct  1  y  related 
to  the  differences  between  the  readability  of  the  procedural 
documents  and  the  reading  ability  of  the  users.  They 
further  concluded  that  this  situation  was  unacceptable  and 
should  be  remedied. 

Mo c  k  o  v  a  k 

In  this  often-cited  1974  study  Mockovak  addressed  the 
concerns  of  Johnson,  Relova,  and  Stafford.  Specifically,  he 
attempted  to  determine  the  reading  demands  of  5G  different 

Force  career  ladders,  the  average  reading  grade  level  of 
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A  i  r 


individuals  in  those  ladders,  and  the  reading’  demands  of 
different  types  of  training'  material  (included  in  this  bread 
category  vere  such  diverse  topics  as  Career  Development 
Coarse  material,  Technical  Orders,  and  resident  training' 
literature)  (Mockovak,  1974:5) 

The  methodology  used  to  acquire  the  above  information 
or.sisted  of  t  vo  steps:  applying'  the  FORCAST  readab  i  1  i  t  y 
formula  to  the  "training  materials"  (this  step  allowed 
Mockovak  to  determine  to  what  RGL  these  materials  vere 
V  r  i  t  t  en  )  ,  and  procuring'  the  reading  grade  i  eve  i  scores  j 
Air  Force  personnel.  Unfortunately,  because  me  reading 
grade  level  scores  of  the  personnel  did  not  exist,  the\  had 
to  be  estimated  by  applying  the  Airman’s  Qualifying  Exam 
( AQE )  scores  to  a  regret,  :o:t  equation  developed  by  Madden 
and  Tupes  (I960). 

As  a  result  of  analyzing  the  data  obtained  from  this 
stud},  Mockovak  concluded  that  both  reading  skills  and 
requirements  varied  widely  among  different  Air  Force  career 
fields.  Additionally,  his  research  revealed  that  in 
approximately  one*  quarter  of  the  Air  Force  career  ladders, 
the  reading  requirement  of  the  material  for  that  ladder  vas 
morr  than  two  grades  higher  than  the  reading  level  of  the 
personnel  assigned  (Mockovak,  1974:8). 
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The  approach  of  Kincaid,  .-\agard,  O’Hara,  and  Cottrell 


(1981)  to  the  subject  of  readability  was  unique.  They 
described  how  the  computer  could  be  used  to  assist  authors 
in  their  efforts  to  improve  the  readability  of  technical 
manuals  and  training'  material.  Specifically,  they  explained 
how  the  computer  could  be  programed  to  identify  uncommon 
words  and  1 ong'  sentences.  They  also  tested  the  computer’s 
abilities  to  edit  difficult  words,  phrases,  and  sentences. 
The  computer  readability  editing  system  ( C'RES  )  that  these 
authors  reported  on  used  the  F 1 esch-K i nca i d  readability 
formula  (also  referred  to  as  the  Kincaid  formula)  to  assess 
the  readability  of  technical  material.  This  formula  was 
chosen  because  it  was  the  Department  of  Defense  standard  <"c  - 
ca  1  r  1  a  t  i  ng'  readab  i  I  i  t  y  . 

K i e  ras 

Douglas  Kieras  also  used  the  computer  to  invest :gatc 
the  readability  of  Technical  Orders.  In  a  1985  article 
entitled,  Improving  the  Comprehensibility  of  a  Simulated 
Technical  Manual,  Kieras  arg'ues  that  using'  a  computer-based 
system  to  improve  the  comprehensibility  of  technical  orders 
makes  sense.  He  cl  aims  that  the  traditional  readability 
formulas  (those  based  on  word  familiarity  and  sentence 
complexity)  which  had  been  developed  and  used  over  the  years 
are  not  adequate  to  address  the  problems  associated  with  the 
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rewrit  in ^  of  documents;.  To  solve  this  dilemma,  Kicras 
ft'enmmemls  using'  a  series  of'  rules  he  developed  called  the 
"comprehensibility  rules."  These  rules,  which  cover  such 
diverse  areas  as  sentence  structure,  references,  textual 
coherence,  and  (4)  textual  organization,  are  designed  to 
make  the  rear i t  i ng  of  technical  orders  easier. 

\s  evidence  for  the  va  1  i  cl  i  t  y  of  this  met  hodo  i  jgy  , 

K i eras  rewrote  technical  order  passages  using  his 
comp  rehens i h  i  1  ity  rules;  he  then  compared  the  t  ime  require 
to  read  tne  old  "poor  ly  written"  passages  with  the  t  inn- 
required  to  read  the  rewritten  passages.  The  results  >f 
t  ii  i  s  experiment  substantiated  the  author’s  contention  that 
the  rewritten  passages  were  easier  to  read.  Additional 
evidence,  which  corroborates  the  value  of  using  Kicras 1 
methodology  to  rewrite  TOs ,  was  the  improvements  in  task 
completion  using'  the  rewritten  material. 

Pa  »•  a  1  1  e  1  Fa c  tors 

This  section  of  the  thesis  is  included  to  provide  the 
reader  with  vital  background  information  relative  to  the 
readabi I ity  of  Air  Force  documents.  The  scope  of  he 
material  presented  is  broad  and  covers  diverse  topic  areas 
including  methodologies  for  assessing  the  reading  grade 
level  of  the  service  member,  the  effectiveness  of 
readabi  1  ity  formulas  at  predict,  ing  comprehension,  and  the 


impact  of  literacy  gaps  on  reading'  comprehens  i  on  scores. 

The  first  of  the  authors  examined  is  William  Mockovak. 

Moekovak 

Mockovak’ s  1974  research  attempted  to  determine  the 
reading'  requirement  level  of  the  training  material  for 
different  Air  Force  career  fields.  Specifically,  he 
examined  a  number  of  readability  "formulas"  to  assess  their 
applicability  for  this  task.  Included  in  this  examination 
were:  Job  Reading  Task  Tests  (not  a  true  readability  test, 

but  rather  a  performance  assessment  technique),  the  Fog 
Count ,  the  CLOZE  technique,  and  the  FORCAST  Formula.  The 
criteria  used  i  n  choosing  which  formula  t.  o  use  included: 
objectivity  of  the  formula,  the  formula’s  validity,  its  ease 
of  use,  its  preparation  costs,  and  the  time  constraints 
associated  with  running  the  tests. 

As  a  result  of  these  constraints  Mockovak  chose  to  use 
the  FORCAST  formula  for  his  analysis  of  the  readability  of 
Air  Force  technical  materials  because  it  was  previously  used 
with  LAS.  Army  training  literature,  it  included  an  effective 
reading  grade  level  range  of  5  -  12.9,  it  *as  easy  to  use, 

and  it  required  relatively  little  computational  time 
(Mockovak,  January  1974:10). 

Another  team  of  investigators  who  were  conducting 
research  on  readability  i n  1974  were  Thomas  Post  and  Harold 
Price. 
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I v_sj _ cu u!  Pr  i  c  e 

Post  and  Pi' ice  reported  on  requirements  and  criteria 
which  the\  felt  should  be  incorporated  by  Navy  technical 
writers  to  improve  the  readability  and  comprehensibility  o 
their  works.  Specifically,  the  recommendations  given  .en 
designed  to  achieve  a  reading'  grade  I  ••  \  <_•  l  of  around  0. 

These  rrcnmmemla! ions,  re iat  ive  to  the  subject  of 
readability,  fell  into  six  general  areas. 

The  first  area  dealt  with  the  need  for  creating’ 
appropriate  paragraph  heading's.  Included  m  this  sec'  ion 
sere  the  recommendations  that  fifty  percent  of  the 
subparagraphs  should  have  lead-ins  or  subheading’s,  and  tn.it 
these  heading’s  should  be  less  than  five  words  in  length. 
\ddit  ional  ly,  this  sect  ion  stressed  the  importance  of 
ensuring  that  all  of  the  material  within  a  paragraph  aas 
consistent  i  t  h  the  paragraph  heading. 

The  second  area  Post  and  Price  commented  on  was  that 
paragraphs  should  be  limited  to  a  few  clearly  ident  i  f  i  ed 
topics.  Recommendations  in  this  area  included  writing  clear 
and  concise  topic  sentences,  limiting'  the  prose  material  of 
a  paragraph  to  three  main  points,  and  presenting  the 
material  in  a  tabular  form  if  five  or  more  main  points  wer** 
t  i >  be  included. 

The  third  area  these  invest  i gators  dealt  with  was  the 
topic  of  words  per  paragraph  and  words  per  sentence.  The 
key  recoinmenciat  ion  given  in  this  area  included:  limit  i  ng’ 
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paragraphs  to  sixty  morels,  limiting  sentences  to  twenty 
words,  changing  complex  and  compound  sentences  to  simple 
sentences,  eliminating  prepositional  phrases  and  unnecessary 
modifiers,  writing  with  an  active  voice  and  using  "peak 
stress  emphasis"  (underline,  italics,  or  other  h i gh 1 i gh t i ng 
techniques)  when  lengthy  paragraphs  could  not  be  avoided. 

The  fourth  readability  area  the  authors  stressed  was 
the  importance  of  eliminating  lengthy  and  unfamiliar  w  ords 
from  text .  These  authors  thought  that  text  designed  for  a 
reading  grade  level  of  0  should  average  about  1.5  syllables 
per  w  o  r  d . 

The  final  two  areas  these  researchers  commented  on  were 
the  importance  of  using  pictures  when  introducing  key 
equipment  and  the  significance  of  page  formats  for  text 
readability  (Post  and  Price,  1974:60). 

Mathews,  Valentine,  and  Se liman 

The  main  purpose  of  this  1978  study  was  to  determine 
the  reading  ability  of  service  applicants  and  selectees,  and 
to  assess  what  relationship,  if  any  existed  between  the 
Armed  Services  Vocational  Aptitude  Battery  t ASYAB )  and 
reading  grade  level  scores.  Many  people  believed  that  the 
ASVAB,  due  to  its  heavy  emphasis  on  verbal  contents,  already 
indirectly  measured  reading  abilities.  This  belief  was 
directly  tested  by  these  researchers  (Mathews  and  others, 
1978:5) . 
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In  designing'  their  tests,  the  authors  used  both  the 
Gat es -MacG in i t i e  Reading'  Tests  Survey  D,  and  the  Neison- 
Denny  Reading’  Test  Form  C  to  establish  reading  grade  level 
scores  for  the  service  members.  They  then  ran  multiple 
correlat ions  between  the  ASVAB  (and  various  sub-test 
combinations  of  the  ASVAB)  and  the  Ga t es -MacG i n i t i e ,  \elson- 
Denny,  and  the  average  reading  grade  level. 

As  a  result  of  this  experimentation,  the  authors 
reached  three  conclusions.  The  medium  Ga t e s -MacG i n i t i e 
reading  grade  level  of  applicants  who  qualified  for  the 
services  was  10.2  (10.9  for  the  Air  Force,  10.5  for  the 
Navy,  and  9.3  for  both  the  Army  and  the  Marines;.  The 
multiple  correlations  between  the  three  ASVAB  subtest 
combination  of  word  knowledge,  numeric  operations,  and 
general  science,  and  the  N’e 1 s on -Denny ,  Gat es -MacG i n i t i e ,  and 
the  average  reading  grade  levels  were  .77,  .80,  and  .8G. 

Their  research  indicated  that  the  ASVAB  was  effectively 
screening  out  most  of  the  service  applicants  with  marginal 
literacy  skills  (Mathews  and  others,  1978:7-11). 


Kern 

In  1980  Kern  attempted  to  answer  three  specific 
questions:  How  effective  are  readability  formulas  at 

predicting  comprehension  for  a  targeted  grade  level0  Does 
rewriting  material  to  a  lower  readability  grade  level 
increase  its  comprehensibility?  How  does  the  requirement  to 
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write  to  a  formula  score  affect  the  production  of  written 
material  (Kern,  1  9  80:2  )'? 

In  conducting  this  investigation  Kern  examined  a  number 
of  readability  formulas,  including  the  Gunning  Fog  Index; 
the  Kincaid  formula;  the  Dale,  Chall  formula;  and  the  Farr, 
Jenkins,  Patterson  formula  to  see  how  well  they  predicted 
reading  grade  level  ( RGL ) ,  He  also  analyzed  recent 
readability  studies  to  see  if  they  could  help  to  provide 
answers  to  his  investigative  questions.  Based  on  this 
research,  Kern  discovered  three  answers  to  his  investigative 
question.  All  readability  formulas  were  unreliable  at 
predicting  the  reading  skill  levels  required  to  understand  a 
passage.  Rewriting  material  to  a  lower  grade  level  does  not 
necessarily  improve  its  comprehensibility.  Requiring  a  text 
to  be  written  to  a  specific  grade  level  score  shifts  the 
attention  of  the  author  away  from  trying  to  communicate  and 
trying  to  reach  a  formula. 

Kniffin,  Stevenson,  Klare,  Entin,  Slaughter,  and  Hooke 

The  principal  focus  of  this  1980  study  was  to  examine 
the  impact  of  the  "literacy  gap"  (which  is  defined  as  the 
difference  between  the  reading  grade  level  of  the  service 
member  and  the  level  of  difficulty  of  the  material  that  the 
serv i remember  was  required  to  comprehend)  upon  the  reading 
comprehension  of  the  service  member. 
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The  methodology  used  by  these  authors  consisted  of 
examining'  three  independent  variables:  Air  Force  personnel 
who  had  reading  grade  levels  of  eight  and  ten,  Air  Force  job 
related  material  (vhich  in  this  case  were  career  development 
course  (CDC)),  written  to  literacy  gaps  of  0,  -2,  and  -4, 

and  reading  times  of  30,  45,  and  60  minutes  (Kniffin,  and 

other  1980:  Abstract).  The  values  assigned  to  these 

variables  were  derived  in  a  number  of  different  ways. 

For  the  career  development  course  material,  the  reading 
grade  levels  were  calculated  by  using  the  Kincaid  formula, 
vhich  is  a  modified  version  of  the  Flesch  Reading  ease 
formula.  (This  formula  is  also  used  by  the  United  States 
Air  Force  to  compute  the  reading  grade  levels  for  its 
technical  orders). 

The  service  members’  RGLs  were  determined  by  using  the 
California  Achievement  Test:  Reading  Level  4.  This  test 
was  chosen  because  norms  were  available  at  grade  equivalents 
for  grade  levels  of  .6  to  13.6.  Additionally,  confidence 
intervals  for  this  test  had  been  determined  at  the  95 
percent,  99.9  percent  and  99.99999  percent  levels. 

After  conducting  the  tests  and  analyzing  the  data, 
these  researchers  reached  numbers  of  interesting 
conclusions:  literacy  gaps  produce  a  small  but  significant 

difference  in  comprehens ion  scores;  increased  reading  time, 
for  the  range  of  times  used  in  the  experiment,  tends  to 
increase  reading  comprehension:  however,  the  relation 
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between  reading1  time  increase  and  comprehension  level 
increase  is  such  that  subjects  given  more  time  learn  less 
efficiently;  and  finally,  of  subjects  who  expressed  a 
preference,  the  more  readable  passage  was  the  one  strongly 
preferred  (Kniffin  and  others,  1980:53). 

Cone  1  us i ons 

Although  numerous  researchers  have  examined  "he 
applicability  of  using  readability  formulas  with  military 
publications,  the  issue  is  still  unresolved.  Both 
dissenters  (Kern,  1980)  and  supporters  (Kincaid  and  others, 
1975)  of  using  readability  formulas  provide  powerful 
arguments  jus*;fying  their  positions. 

In  terms  of  today's  procedures,  the  supporters  of  using 
readability  formulas  have  seen  their  arguments  prevail.  The 
specifics  of  the  current  method  for  examining  the 
readability  of  military  publications  are  discussed  in  the 
next  chapter. 
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The  problem  of  readability  and  its  usefulness  in 
military  publications  has  been  a  concern  of  the  United 
States  government  for  nearly  40  years.  This  chapter  will 
analyze  current  Air  Force  policy  relative  to  the 
readability  of  its  publications.  This  presentation  can  be 
divided  into  two  stibsect  ions  .  The  first  subsection  analyzes 
articles  focusing  on  the  readability  of  regulations.  The 
second  subsection  examines  articles  exploring  the  topic  of. 
readability  and  its  relationship  to  technical  orders  and 
procedural  manuals. 

Regu 1  at  ions 

Two  Air  Force  regulations  (AFRs)  serve  as  the  primary 
source  documents  for  the  readability  of  the  Air  Force- 
regulations.  AFR  5-1,  Air  Force  Publication  Management 
Program  (April,  1986),  explains  the  general  Air  Force 
policies  for  producing  a  readable  publication.  AFR  5-8, 
Preparing  Air  Force  Publication  (9  April,  1986),  examines 
those  procedures  at  a  greater  depth,  and  suggests  specific 
techniques  for  implementing  the  policy  guidance  of  AFR  5-1. 

AFR  5-1  states  that  for  reg'ulations  to  be  readable  they 
must,  at  a  minimum,  be  written  to  comply  with  the  plain 
"English  Standards  "  These  standards  seek  to  answer  the 


following  11  questions  about  the  regulation. 


(  1 )  Is  it 


primarily  written  in  the  active  voice?  (2)  Is  the  style 
used  by  the  text  uncluttered?  (3)  Are  the  sentences  clearly 
constructed  and  grammatically  correct?  (-4)  Do  the  sentences 
average  fever  than  21  words?  (5)  Are  the  sentences  composed 
of  simple,  familiar  words  or  abstract  words,  unnecessarily 
technical  words  and  jarg'on?  (6)  Are  there  illogical  or 
inconsistent  shifts  in  the  point  of  view  (tense,  person,  cr 
voice)  within  a  paragraph?  (7)  Is  the  material  preferred  in 
a  logical  sequence?  (8)  Is  each  paragraph  limited  to  one 
topic-?  (9)  Is  each  paragraph  as  brief  as  possible-.1  (10) 

Are  there  as  many  main  paragraphs  as  possible  rather  than 
drawn-out  subparagraphing-?  (11)  Are  the  titles  for  parts, 
chapters,  sections,  and  paragraph  descriptive  and  are  ; hey 
designed  to  ensure  that  the  issue  of  readability  is 
considered  in  the  writing  of  any  regulation? 

Air  Force  Regulation,  (AFR)  5-8,  Preparing  Air  Force 
Pub  1 i ca t  i ons ,  is  the  best  source  document  available  to  the 
project  officer  who  prepares,  manages,  reviews,  approves,  or 
uses  Air  Force  publications.  This  regulation  not  only 
provides  general  guidelines  for  improving  the  readability  of 
\ir  Force  regulations,  but  it  also  gives  specific 
recommendat  ions  to  authors  as  to  how  they  can  produce  a  more 
understandable  publication.  Unfortunately,  because  this 
information  is  scattered  throughout  the  text  of  this 
regulation,  it  is  often  difficult  to  find.  To  simplify  this 
procedure,  pertinent  areas  will  be  summarized  below. 
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Section  2-31,  entitled  Improving  Readability,  provide 
t vo  general  recommendations  for  obtaining  a  more  readable 
writing  style.  The  first  recommendation  is  to  replace 
multisyllabic  words  with  shorter  words.  The  second 
recommendat ion  states  that  in  order  to  achieve  a  more 
readable  text,  one  should  simply  rewrite  the  publication. 
Nowhere  in  this  section  is  it  mentioned  that  the  topic  of 
readability  is  discussed  elsewhere  in  the  regulation. 

Section  2-3,  Preparing  the  First  Draft,  argues  that 
traditional  readability  factors  should  be  considered  in 
writing  regulations.  This  section  emphasizes  the  import  an 
of  using  the  following  factors  to  achieve  a  readable  wr  i  l  i 
style:  (1)  language  familiar  to  the  intended  users,  (2) 

short  sentence  arid  paragraph  structure,  and  (3)  unambigunu 
language.  Additionally,  and  perhaps  most  importantly,  thi 
section  directs  the  reader  to  attachment  two  of  the 
regulation  for  more  information. 

Attachment  2,  entitled  Procedures  for  Reviewing  and 
Ed i t  i ng  Draft  s ,  is  divided  into  a  number  of  different 
sections.  Sect  ion  C,  which  examines  the  writing  style  of 
Air  Force  Regulat ions,  is  of  part icular  interest  to  the 
readability  researcher.  This  sect  ion  has  been  subdivided 
into  a  number  of  topics,  three  of  which  deal  directly  with 
the  subject  of  readability.  These  pertinent  topic  ai uas 
include:  (1)  sentence  and  paragraph  length,  (2)  active 


voice,  and  (3)  techniques  for  improving  the  readability  of 
reg'u  1  at  i  ons  . 

The  first  section  addresses  the  subject  of  sentence  and 
paragraph  length.  It  recommends  that  sentences  which  make 
up  the  paragraphs  of  regulations  be  no  more  than  20  words 
long.  It  also  suggests  that  authors  keep  their  paragraphs 
short  (preferably  to  four  or  five  sentences)  so  that  the 
readability  of  those  paragraphs  can  be  substantially 
i mp roved . 

Section  two  advocates  using  the  active  voice  when 
writing  to  improve  the  under s t andab i 1 i t y  of  regulations. 
Active  voice  is  more  natural  and  direct  than  the  passive 
voice,  and  it  is  easier  for  the  reader  to  comprehend 
information  presented  in  this  format. 

The  final  section  deals  with  other  techiiiques  that  can 
be  used  to  improve  the  readability  of  regulations.  These 
techniques  consist  of:  substituting  one - s y 1 1 ab 1 e  words  for 

longer  words  (see  appendix  number  one  for  a  sample  of  these 
proposed  substitutions),  writing  directly  to  the  reader  by- 
attempting  to  use  the  second  person,  being  concise,  not 
hiding  the  main  ideas  in  the  body  of  a  paragraph,  not  using’ 
smothered  verbs,  not  splitting  predicates  with  prepositional 
phases  and  clauses,  not  using  the  slash  ("/")  to  combine 
ordinary  terms,  and  using  parallel  construction  when 
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p  o  s  s  i  b  1  e  . 


Ad  i  i  t  i  ona  1  1  y  ,  Change  1  to  attachment  2  pro',  ides  a 
listing  of  words  designed  specifically  to  produce  a  mc-e 
readable  publication.  A  sample  listing  of  these  words  is 
provided  in  Appendix  A. 

Although  the  Air  Force  provides  extensive  guidance  on 
techniques  for  producing  readable  regulations,  little 
research  exists  which  examines  the  effectiveness  of  that 
guidance  or  the  success  that  Air  Force  writers  have  had  in 
implementing  that  guidance.  One  notable  exception  to  this 
situation  is  the  1979  study  of  Hooke,  DeLeo,  and  Slaughter, 
discussed  i u  chap  ter  4. 

In  general,  the  Air  Force  approach  for  ensuring  the 
readability  of  its  non-Techn i ca 1  Order  publications  seems 
uncertain.  Information  on  this  topic  is  scattered  among 
numerous  regulations,  and  the  advice  that  is  given  does  not 
appear  to  have  been  empirically  tested. 

Additionally,  this  researcher  found  no  official 
justification  for  the  recommendations  that  were  given,  nor 
any  precise  evidence  indicating  how  well  those 
recommendations  were  being  followed.  It  is  interesting  to 
note  that  at  one  time  the  Air  Force  required  its  regulations 
to  meet  a  readability  standard.  Why  this  requirement  was 
abandoned,  however,  is  not  clear.  The  Air  Force  guidance  on 
producing  readable  Technical  Orders,  however,  is  very  clear. 


Procedural  Manua  I  s /Techti  i  ca  I  Orders 

The  problems  of  writing  Technical  Orders  ( TOs  )  and 
procedural  manuals  are  more  complicated  than  those  of  any 
other  document.  Problems  arise  not  only  because  technical 
information  itself  is  expanding'  at  a  tremendous  rate,  but 
also  because  it  comes  from  diverse  sources.  The  scientist 
or  engineer  who. creates  the  product,  as  veil  as  the  peopl  > 
who  manufacture,  inspect,  service,  and  use  the  product 
possess  complex  informat  ion  which  must  to  be  included 
(AEC'MA,  1986:  iii).  Additionally,  this  information  must  be 
written  in  such  a  way  that  it  is  readable  by  its  target 
audience.  To  ensure  that,  this  requirement  is  met,  the  L' .  S  . 
Air  Force  mandates  that  its  technical  orders  and  procedural 
manuals  adhere  to  the  requirements  specified  in  AFR  8-2,  A i r 
Force  Technical  Order  System  (17  April,  1907). 

AFR  8-2,  the  Air  Force’s  governing  document  on 
Technical  Orders,  requires  that  numerous  actions  occur  prior 
to  the  publishing  of  the  Technical  Order.  Specifically,  the 
process  of  producing  a  readable  TO  begins  with  the 
contractor  agreeing  to  write  the  narrative  material  of  th- 
Technical  Order  to  the  reading  grade  level  ( RGL )  of  the 
target  audience.  Excluded  from  this  RGL  requirement, 
however,  are  TOs  that  contain  "little  if  any  narrative" 
text,  such  as: 

(1)  Airercift  Flight  Manuals  (-1  Series) 

(2)  Job  Guide  Manuals  (-2  JG  Series) 
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(?>)  Illustrated  Parts  Breakdown  (IPB)  Manuals  (-5 
S  i  ■  r  i  e  s  ) 

(4)  Basic  Weight  Checklist  and  Loading  Da* a  Manuals  (- 
5  Series) 

(5)  Cargo  Aircraft  Loading  Manuals  (-9  Series) 

{6)  Non-Nuclear  Munitions  Delivery  Manuals  (-34 
Series) 

(7)  Acceptance  and/or  Functional  Check  Flight  Manual-. 

( -  0  C  F -  1  Series) 

(3)  Work  Cnit  Code  Manuals  (-03  Series) 

(3)  List,  of  Applicable  Publications  (L«»APS)  (-0  1 
S  cries  ) 

(10)  Aircraft  Inspection  Manuals  and  Associated 

Wo  rkciirds /Wo  r  k.shee  t  s  (-G  and  GWC/-GWS  Series) 

(11)  Wiring/Schematic  Diagram  Manuals. 

\dd  i  t  ional  iy,  APR  8-2  directs  contractors  to  i  den  t  i  t'y  h  i  •:  h 
Air  Force  Special  ty  Code  (Al'SC)  would  be  ro<ju  i  red  to  use  t  he 
To,  to  determine  their  KGL ,  and  to  write  the  Technical  Order 
to  that  level.  This  regulation  also  states  that  informal  ion 
regarding  the  RGL  for  each  AFSC  is  available  in  M I L - STD- 1 7  a  2 
( Military  Standard  Reading  Level  Requirements  for  the 
Preparat  ion  of  Technical  Orders)  .  Unfortunately,  when  i h  i  s 
researcher  tried  to  verify  that  information  Lv  obtaining  a 
copy  of  M I  L  -  STD-  1  7  5  2  ,  he  found  that,  it  had  been  ch  let  ed. 

The  not  ice  cancel  1  ing  it  stated  that  Technical  orders  should 
he  written  to  an  RGL,  of  9.  Add  i  t  ional  ly,  the  cancel  1  at  ion 
not  ice  recommends  using  the  Assoc  iat  ion  Kuropeene  Des 
Const  ru  lours  Do  Mat  (’riel  Aerospat  i  a  1  (  AECMA )  document 

number  P3C-B5-1G50B  (19BG)  ent  it  led  AECMA  Simplifie d 
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Fti  g  Mshi  Guide  for  the  Preparation  of  Aircraft  Maintenance 

Documentation  in  the  International _ Aerospace  Mainte nance 

Language  as  a  tool  to  assist  the  writers  of  technical  orders 
in  their  efforts  to  achieve  a  ninth  g'rade  reading’  level 
( MIL-STD- 1 752  Notice  1,  1988).  In  assessing'  the  utility  of 
this  document  as  an  aid  to  writing’  Technical  Orders  it  is 
important  to  understand  the  document’s  origins. 

\ECM PSC- 8 5  -  1  6 5 9 8  was  born  out  of  a  need  in  t he 
aerospace  industry  for  clear  and  concise  communication  of 
complex  maintenance  information.  As  a  starting'  point  for 
producing'  this  document,  the  Association  of  European 
Airlines  asked  airplane  manufacturers  to  investigate 
readability  criteria  for  maintenance  documentation  within 
'he  civilian  aircraft  industry.  The  response  they  received 
was  overwhelming’,  with  companies  from  Italy,  France,  "•%'.*>  t 
Germany,  England,  Holland,  and  the  Un  i  t  eel  States  all 
choosing  to  work  on  this  project  (Appendix  B  provides  a 
complete  list  of  participating  companies). 

The  output  this  combined  effort  produced  is  unique  for 
a  number  of  reasons:  it  establ ishes  one  word  for  a 

par  t  :  "  u !  ;*.r  notion,  thus  eliminating  the  problem  of  diffb -rent 
■*i>rdi  being  used  to  define  the  same  thing,  it  rigidly 
defines  the  meaning  of  the  word,  and  it  establishes  a  set  of 
rules  designed  to  simplify  any  author’s  writing  -style  . 

The  rules  developed  to  achieve  this  simplified  writing 
style*  fall  under  eight  broad  categories:  (1)  words,  (2)  the- 


g r  o u pi:: g  o f  ' %  o r  <.! s  ,  (  3  )  v erbs  ,  (  4  )  se n t  ences  ,  (  5  )  p r o c e >.1  r.r<«s  . 

(6)  clescript  ive  writ  ing',  (7)  warning's  and  cautions,  and  (8) 
punctuation  (AEC’MA,  1986:1-0-1  through  2-2-93).  These-  rules 
vere  designed  not  to  achieve  a  specific  reading'  grade  level, 
but  rather  to  simplify  an  author’s  writing'.  However, 
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387  8  4B  (1(1  April,  1983)  Mi  1  i  t  ary  Spec  i  f  i  cat  i  on  Manna  1  s  , 
Technical  :  General  S  t  y  I  e  and  Format  Requirement  s  t  see  i-v, 
this  task  is  supposedly  achieved. 


M I L  -M-  3  87  3  4B  was  designed  to  be  used  by  all  depart  mot.t  s 
and  agencies  of  'he  Department  of  Defense  ( DOD ) .  Its 
primary  purpose  is  to  establish  the  general  style  and  f •  rmn‘ 
requirements  for  the  preparation  of  standard  technical 
manuals  and  changes  thereto  ( M  II.-M-38784B ,  1983:1). 

Included  in  the  general  style  category  is  the  topic  of 
readability.  This  document  requires  that  "technical 
publications  be  written  at  a  Reading  Grade  Level  ( RGL ) 
commensurate  sith  the  capability  of  the-  target  audience" 

(  M  1 1.  -  .1 8  /  8  4  8  ,  1  9  8  3:15).  It  states  that  when  t  e  s  1  e  d  ,  t  i  n  •  - 
I  ecnnica!  document  t  s  must  not  have  an  over  a  1  1  grade  level 
which  exceeds  the  appropriate  RGL  by  more  than  1.0  grade 
levels.  Add  i  t  iona  I  ly,  the*  grade  level  of  each  tested  sample 
i  .->  not  ti>  exceed  the  appropriate  RGL  by  more  than  3.0  grade 
levels.  To  ensure  that  RGL  requirements  of  this 
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ipec  i  f  i  ca  !  ion  are  met,  this  document  man  da  t  t.  a  that  a 
validation  check  occur.  The  specific  nature  of  this 
extensive  check  can  be  found  in  Appendix  C'.  Unlike  the 
topic  of  readability  and  regulations,  the  topic  of 
readability  and  its  relationship  to  the  military  procedural 
munua 1 /Teehn i ca 1  Order  has  been  extensively  invest  igate  i. 

N’<v.»  that  the  procedures  for  producing  a  readab  1  •'* 
technical  order  have  been  clearly  provided,  the  iu-x‘  step  is 
to  examine  the  Air  Force’s  rationale  for  following  those 
procedures.  Th<-  Air  Force  selected  the  ninth  grade  as  t  he 
targeted  RGI,  because  most  individuals  entering  the  Air  Force 
had  at  least  this  RGL ,  and  for  those  that  did  not  ,  remedial 
reading’  training’  programs  are  available  at  most  bases  to 
increase  reading  skill  levels  (Mockovak,  107  4:4  )  .  I'  was 
also  thought  to  be  more  economical  to  write  to  a  standard 
ninth  grade  level  as  opposed  to  reevaluating  the  read i ng 
requirement  levels  for  various  Air  Force  Specialty  Codes 
(  AFSC' )  . 

The  Air  Force’s  stated  rationale  for  excluding  certain 
series  of  Technical  Orders  from  mce*  ing  the  reading  grade 
level  requirement  is  based  on  the  assumption  that  those  TOs 
contain  little  if  an y  na  r  r a  t  i  ve  text.  Be  e  a u  s  e  t e x *  i  s 
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The  Air  Force  decision  to  recommend  the  use  of  AFCMA  to 


achieve  a  more  readable  writing  style  for  its  Technical 
Orders  appears  to  be  based  on  a  number  of  diverse  factors 
including  the  fact  that  AECN1A  was  endorsed  by  most  of  the 
free  world’s  major  commercial  aerospace  companies  (AEC'MA, 
1986).  Because  AECMA  provides  specific  rules  for 
simplifying'  writing  style,  this  document  was  viewed,  by  the 
\  I r  Force,  as  a  valuable  first  step  in  standardizing  their 
Technical  Orders. 

The  DOD  elected  to  use  the  Kincaid  readability  formula 
to  assess  the  readability  of  its  Technical  Orders  for  a 
number  of  reasons.  The  formula  was  developed  using  military 
personnel  as  subjects  and  using  military  technical 
materials,  and  it  was  thought  to  have  appropriate  norms. 
Flesch  type  formulas  (which  include  the  Kincaid  formula) 
have  been  used  extensively  in  examining  the  issue  of 
readability,  and  thus  a  gretiter  number  of  people  are  liheiy 
to  be  familiar  with  their  application.  The  word  and 
syllable  counts  required  to  use  the  Kincaid  formula  can  be 
easily  determined,  and  finally,  the  Kincaid  formula  can  be 
used  to  compute  the  reading  grade  level  of  a  document  either 
manually  or  automat  ically  without  too  much  inconvenience  or 
cost  ( Kn i f  f i n  ,  1981:3). 

The  technique  for  achieving  readable  technical  orders 
and  the  justifications  for  using  this  technique  at  first 
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glance  seem  solid.  However,  upon  close  examination  numerous 
potential  problem  areas  begin  to  appear. 

The  first  of  these  potential  trouble  areas  is  the 
requirement  that  the  Technical  Orders  be  written  to  a  ninth 
grade  reading  level.  Numerous  researchers  have  indicated 
that  the  requirement  is  unnecessarily  restrictive  and  may  be 
extremely  costly  to  determine  (Johnson  and  others,  1972: on; 
Kniffin,  undated :  9-4 )  .  Additionally,  this  researcher  ’'ound 
no  studies  examining  the  impact  on  the  readability  of  a 
document  if  the  material  were  written  to  a  RGL  subs t ant  ially 
below  the  RGL  of  its  targeted  audience.  It.  is  conceivable 
that  a  lower  RGL  could  make  the  material  too  simple  to  t  h :  • 
reader  and  thus  less  valuable  as  a  tool  for  transferring 
info  rma  t  ion. 

The  rationale  for  excluding  certain  types  of  Technical 
orders  from  meet  i ng  the  ninth  grade  reading  level 
requirement  also  seems  sound;  however,  once  again  a  close 
examination  of  the  logic  behind  this  choice  reveals 
problems.  The1  first  and  most  obvious  discrepancy  is  the 
incorrect  fundamental  assumption  on  which  the  decision 
rest  s,  the  as  sump t  ion  that  -1  Series  of  Technical  Orders 
contains  little  if  any  narrative  text.  This  assumption  is 
obviously  wrong,  £is  the  examination  of  any  aircraft  -1 
clearly  demonstrates. 

Another  possible  trouble  area  that  contractors  face  in 


ensuring  that  their  Technical  Orders  are  readable  is  the 


recommendation  to  use  the  AECMA  Guide  for  the  Prepara:  i  or.  of 
Aircraft  Maintenance  Documentation  in  the  I n t e r na t i ona i 
Maintenance  Language  as  a  guide  to  reach  the  ninth  grade 
reading  level.  The  problem  with  this  advice  is  twofold; 
first,  AEC'MA  was  designed  specifically  as  an  aid  for  writing 
Technical  Orders  relating  to  the  aerospace  industry,  and  it 
may  not  be  applicable  for  other  Air  Force  Technical  0:*de  r;:  . 
Secondly,  although  the  recommendations  that  AECMA  makes 
appear  logical,  they  do  not  seem  to  have  been  empirically 
tested. 

The  last  step  that  the  Air  Force  requires  in  order  t  > 
check  the  readability  of  its  Technical  Orders  contains  a: 
least  three  potential  problems.  The  first,  problem  is  that 
the  Kincaid  formula  was  developed  as  a  result  of  sludit  s  ■  n 
Navy  personnel  and  Navy  Technical  Orders,  and  it  may  no'  be 
an  appropriate  test  for  the  readab i 1  it\  of  Air  Force 
material.  Secondly,  even  if  future  studies  validate-  the  use 
of  the  Kincaid  formula,  the  Air  Force’s  decision  to  exclude 
certain  mu  1 t  i - sy 1  lab i c  technical  words  from  calculation  when 
computing  the  Technical  Order’s  reading  grade  level 
invalidates  the  use  of  this  formula,  thus  causing  the 
reading  grade  level  score  to  appear  lower  than  it  aciually 
is.  F  i  na  1  I  y  ,  Kincaid’s  (  and  o  t  hers  )  1975  article  shot,  s  t  he 
original  Flesch  Reading  F.ase  formula  calculated  in  a 
different  manner  than  that  found  in  any  other  source  (Klare, 
1903:59;  Flesch,  1971:247-251).  The  formula  as  displayed  by 


Kincaid  and  others  (1975)  is  as  computed  as  follows : 

RE  =  206.835  -  1.015  ( vords/sent ence )  -  .836  ( s y 1  I ab 1 es/ 1 00 

words )  . 

However,  the  formula  that  appears  in  Flesch  (1974)  is 
slightly  different.  This  formula  is  calculated  as  follows: 
RE  =  206.835  -1.015  (words/sentence)  -.846  ( sy 1 1 ab 1 es / 1 00 

words ) . 

The  impact  of  the  .01  difference  between  the  two  formulas  is 
yet  to  be  determined,  although  logic  would  indicate  t  ha  t 
this  influence  is  probably  minimal. 

Co  tic  1  us  ions 

The  recent  decisions  to  change  the  procedures  for 
determining  the  Reading  Grade  Levels  of  Technical  Orders,  as 
well  as  the  decision  to  exclude  regulations  from  meeting 
readability  requirements,  suggest  that  there  is  some 
bewilderment  about  the  usefulness  of  readability  formulas 
within  the  Department  Of  Defense.  This  confusion  is  further 
evident  by  the  recommendation  to  adopt  the  practices 
indicated  in  the  AECMA  document.  This  recommendation 
appears  to  imply  that  semantics,  as  well  its  syllables  per 
word  and  words  per  sentence,  are  all  important  in 
determining'  a  publication’s  readability. 

Addif  ionally,  the  technique  employed  to  determine  the 
Reading  Grade  Level  of  TOs  boars  reevaluation  in  lieu  of  the 
findings  in  this  chapter. 


6.  Cone  1  us  i  otis  and  Recommendations 


Cone  1  us  ions 

Based  on  information  presented  in  the  previous  chapters 
it  is  reasonable  to  conclude  that  the  Air  Force  does  not 
have  a  clearly  defined  methodology  in  place  which  can 
confidently  assure  the  readability  of  its  publications.  The 
current  Air  Force  procedures  employed  to  address  this 
problem  are  inadequate.  Revaluation  needs  encompass  hot  hi 
the  readability  of  Technical  Orders  and  the  readability  of 
other  Air  Force  publications. 

Regarding  the  topic  of  readability  of  its  Technical 
Orders,  the  Air  Force  should  reexamine  the  requirement  to 
write  the  material  to  a  ninth  grade  reading  level.  To 
automatically  assume  this  requirement  will  save  money 
without  empirically  testing  this  assumption  seems  unwise. 

It  is  perhaps  additionally  imprudent  not  to  investigate  ’he 
impact  of  producing  technical  reading  material  significantly 
below  the  reading  grade  level  of  the  intended  user. 

The  Air  Force  rationale  for  excluding  certain  Technical 
Orders  from  meeting  reading  grade  level  requirements  aUu 
needs  to  be  reexamined.  This  requirement,  which  is  based 
specifically  on  the  belief  that  these  TOs  contain  little  if 
any  narrative  text,  clearly  does  not  app 1 y  to  the  -1 
series  of  Technical  Orders,  and  may  not  in  fact  apply  to  the 
other  excluded  documents  as  well. 
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Another  area  that  the  Air  Force  should  reconsider  is 


the  recommendation  to  use  AECMA  as  a  tool  to  assist  the 
writers  of  Technical  Orders  in  their  efforts  to  achieve  a 
ninth  grade  reading'  level-  This  suggestion  is  premature  for 
a  number  of  reasons.  First,  AECMA  was  designed  to  simplify 
a  writing  style,  not  to  achieve  a  specific  reading  grade 
level.  Although  this  document  can  be  used  as  an  effect  i  v 
aid  to  w r i t  ing  to  a  9  RGL  it,  does  not  guarantee  those 
results.  In  fact  many  experts  on  readability  caution 
against  writing  to  a  formula,  as  this  technique  often 
produces  a  document  which  is  even  less  readable  ( Kincaid,  and 
o  t,  he  r  s  ,  19  6  7:7). 

Additionally,  because  AECMA  was  originally  developed  to 
he  used  in  the  aerospace  industry,  it  contains  terminology 
peculiar  to  that  industry.  Therefore  the  suggestion  f.htt  i; 
is  applicable  to  all  Air  Force  TOs  is  ur.substant  iated  and 
bears  further  investigation. 

Equally  unsubstantiated  is  the  readability  formula  that 
the  Air  Force  uses  (the  Kincaid  formula).  This  formula 
should  be  evaluated  to  determine  if  it  is  applicable  to  Air 
Force  Technical  Orders  and  to  ascertain  the  magnitude-  of  r  he 
impact  of  the  Air  Force  decision  to  exclude  certain 
technical  words  from  being  calculated  (by  the  formula)  when 
computing  a  text’s  reading  grade  level.  The  discrepancy 
which  exists  between  the  original  Flesch  formula,  as 
reported  by  Flesch  (1975),  and  the  slightly  different 
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version  of  the  formula  used  by  Kincaid  in  deriving  his 
formula  should  also  be  resolved. 

Although  the  Air  Force  policy  for  establishing'  the 
readability  of  its  TOs  appears  inadequate,  at  least  a 
formally  stated  policy  exists.  This  is  not  the  case  with 
regulations  and  other  Air  Force  publications.  For  these 
documents  the  Air  Force  advice  on  obtaining  a  readable  text 
is  poorly  structured  and  difficult  to  find.  The  guidance 
that  is  given  appears  sound,  but  unfortunately  it  does  net 
seem  to  have  been  empirically  tested. 

Conceivably  the  confusion  that  is  apparent  ir  the 
methodology  and  logic  of  applying*  readability  formulas  to 
Air  Force  documents  is  indicative  of  the  effectiveness  and 
values  of  readability  formulas  as  a  whole.  While'  it  would 
be  improper  to  deny  that  readability  formulas  have  provided 
meaningful  assistance  in  the  area  of  producing  a  readable 
document,  it  seems  clear  that  other  aspects  of  style  are 
important  and  should  be  considered  when  writing  to  produce  a 
readable  document  (Vaughan  and  Ferkinhoff,  1-89:8). 

To  rectify  this  problem,  and  the  others  previously 
ment  ioned,  additional  research  should  be  conducted.  This 
research  may  be  costly  but  the  results  obtained  might  easily 
make  the  investment  worth  while. 
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Recommendat  ions 


Specific  recommendations  for  future  research  include: 

1.  Determining  if  the  ninth  grade  reading  level  is  really 
the  most  desirable  level  for  Air  Force  Technical  Orders 
to  be  written.  Included  in  this  analysis  should  be  both 
cost  analyses,  and  more  importantly,  effectiveness 

ana  lyses . 

2.  Reassessing  the  decision  to  exclude  certain  Technical 
Orders  from  meet ing  reading  level  requirements. 

Certainly  any  document  which  affects  aircraft 
operations,  aircraft  inspection,  or  munitions  handling 
needs  to  be  written  in  a  readable  form. 

3.  Conducting  a  study  to  determine  if  the  Kincaid  formula 
is  really  the  best  evaluative  tool  for  determining 
t  he  re  a d  a  b i  I  i t  y  of  Air  Force  Technical  O r  d e  r  s . 

■1  .  Analyzing  the  effectiveness  of  «•  !r.=  AF.CMA  as  a  tool  f  >r 
simplifying  the  rewriting  of  material  to  a  specified 
grade  level . 

5.  Developing  guidance  relative  to  the  readability  of 
regulations  and  other  military  documents. 
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\ppendix  A:  AFR  5-8  Listing  of 
Simplifying  Words  and  Phrases 


ac  c  omp 1  i sh 
accomplish  (a  form) 


according'!  y 

achieve  the  max :  ,ium 
results 

actual 

a  d  j  a  c  e  n  t  t  u 

aclv  i  s  e 

afford  an  opportunity 

after  a  thoroug'h  review 
of  your  report,  this 
headquarters  approves 
the  recommendation 
contained  therein 

all  information  will  be 
furnished  promptly  to 
the  commander 

all  these  items  will  be 
broken  down  in 
separate  categories 

a  1  ong  t  he  1  i  lies  of 

ame 1 iorate 

and/or 

append 

appropriate 


TRY  THIS 

do 

prepare  ,  comp>  1  et  e  ,  p  roduce  , 

fill  in,  make  out ,  carry 
out 

so,  therefore 
get  the  most  from 

real 

next  to 

tell,  inform 

a  1 1 ow ,  let 

we  approve  your 
recommendat ion 

inform  the  commander 
promp  t  1 y 

list  t  he  s  e  i t  ems  by 
cat  egory 

like,  similar 
improve 

o  r ,  and  (not  both) 

add,  attach 

sui table,  pert  i n e n t  , 

relevant  ,  proper,  right 
or  delete  it) 


GO 


are  desirous  of 


w  an  t  t  o 


are  in  receipt  of 

as  a  means  of 

as  prescribed  by 

as  to  your  request, 
let  ter 

ascertain 

clS  S  I  3  t 

aSb u  r e 


at  .a  later  date 
at  1  1  times 
at  a  n  early  d  a  t  e 


r  e  c  e  i  v  e  d 


under 

about  your  request,  letter 

f i nd  out,  1  earn 
help,  aid 

to  make  sure,  ens u r  :■ , 
insure,  (the  correct 
meaning'  is  none  of 
these,  b u •  t  h e  v  jr d 
is  often  misused  for 
o  n  e  o  f  t  h  e  s  e  ) 

later 

a  1  va y s 

soon,  shortly  »  or  give  the 
date) 


a t  tile  present  time 

at  t  h  I  .->  t  i  m  e 

attached  hereto  (or 
he  rev i t  h ) 

attempts  to 

attention  is  invited  to 
a  u  gm  e  n t 

b i annua  1 

biennial 

by  means  < f 

b y  virtue  o f 

c-cipab  i  1  i  t  y  (  hits  the 
capability  t  o ) 


nov,  currently,  ai  present 
present  1 y 

nov 

attached,  attached  to  this 
here  are 

tries 

see,  please  note 

increase,  extend,  _*n  I  urge  , 
e  x pand 


t  v  i  c  e 

a  %ear 

once 

in  t  vo 

by,  v 

i  t  h 

because 

can 
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be  careful,  take  care 


care  should  be  taken 
close  proximity 
cognizant  of 

c  omb i ne 
commence 

com  m  e n surat e 

c  omp 1 y  v i  t  h 

o oncern  i  n g 
cone  1 ude 

consider  favorably 

consideration  should  be 
given  to  the  fact  that 

e oust i tut  e( s  ) 

contained  in 

c  on  t  a  i  n i ng 

contains  a 

deemed  to  be 

d  e  t  a i led 

de t  e rm i ne 

d  e  v  e  1  o  p 

disseminat e 

do  not  gi  ve  rise  to 
due  in  large  measure 

o 


close,  near 

avare  of,  knov ,  hav^> 

knowledge  of,  under¬ 
stand,  comprehend, 
appreciate,  alert  to 

join 

beg  i  n 

equal  to,  corresponding 
to,  to  agree  with 

carry  out,  meet,  satisfy, 
execut e 

about ,  on 

end,  close 

approve 

note  t  ha  t 

are  (is) 
i  n 

has,  that  have,  etc. 
has 

cons i dered 
more,  full 
dec i de ,  find 
make,  grow 

issue,  get  out,  circulate 
d i s  t  ribute 

do  not  app 1 y 

because,  due  to 


du«-  t 


the  fact  that 


because  of,  since,  hence 


during  the  periods  when 
eche 1 ons 

effect  an  improvement 

emp 1 oy 

ensure 

equ i t  ab 1 e 

e  s  t  a  b  1  is  h 

every  effort  will  be  made 
e  v idenees 
evident 
e  x  p  e  d  i  t  e 

expense 

experience  has  indicated  tha 
e  x  p  i  a  i  n 
f  a  c  i  i  i  t  a  r  e 

f  i  na  l 
f  i  n  a  1  i  z  e 

for  the  purpose  of 
for  the  reason  that 
forvar d 

fullest  possible  extent 
f une t  ions 

gained  from  the  following 
source 

iteiie  rat  e 


when 
levels 
i mp  rove 
use 

make  sure 
fair 

set  up 

I  or  we  .ill  try 

facts 

clear 

hasten,  speed,  c;o 

p  r  o  m  p  t  1  y  ,  m  a  k  e  .•  a  % 

cost,  f  ee ,  price 

1  earned 

‘  ?  w 

ease,  c  1  e a r ,  help,  ail, 
free,  make  easy 

1  as  t 

comp  1 e  t  e ,  finish, 

cone  1 ude ,  se  t  t  1 e 

for,  to 

since,  because 
send 

fully,  to  (he  maximum 
works 

obtained,  learned 
p  r  o  d  u  c  e 
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•q'ive  consideration  to 


cons i der 


govern 

held  a  mee  t  i  ng’ 
hereby 

hereby  advise 
herein,  hereinafter 
herein  set  down 

LISTING  OF  NONSEXIST  TERMS  FOR 

INSTEAD  OF 
cha i rman 

c  1  er  gymait  (  en  ) 

C'ong'res  sman  (  en  ) 

crevman ( en ) 
draft  sman ( en ) 
f  oremati '  en  ) 

journeyman ( en } 
j  u  ryrnat.  (  en  ) 

1 ayman ( en ) 
man-hour ( s  ) 


rule 
me  t 

by  this 

advise,  advise  by  !his 
here 

listed  here,  stated  here 
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chairperson;  chair;  head 
of  the  c omm i 1 1  ee 

c  1  e rg'y  member  (  s  )  ; 

member! s)  of  the  clergy 

member! s  j  of  Congress; 
Congressmen  and 
Congre s  s women 

e  revmembe r  s 

drafter! s  ) 

supervisor! s  )  ; 

superintendent;  shop 
ch i e  f ( s ) 

journey  worker! s) 

jury  member! s ) ;  member! s) 
of  the  j  u r y 

1  ay  person ( s , 

work  hour(s) 


mank i nd 


human  race 


manmade 

manned  air  e  r  a  f '  t 

maun i ng 

Nat  ional  Guardsman  (  en  ) 

po  1  i  e ('man  (  en  ) 
p  re s s man ( en ) 
repa i rman ( en ) 
icr\  i  c email  <  en  ) 

spokesman ( en  ) 

s  f  e  v  a  r  d  <  s  )  ;  s  f  e  va  rdess  {  e  s  ) 

unmanned  a  i  re  r a  f  f 

unde rmanned 
s  a  t  <;  hmau  (  en  ) 
s  i  i' eman  |  en  ) 

virlunen  ’  s  c  ompen  s  a  f  i  on 


machine  made;  synt  h"t  i  <•  ; 
artificial;  simulated 

occupied  air  -rat' *  ;  pi  !ot--d 
aircraft 

staffing 

member( s)  of  the  Nat  ional 
Guard 

police  o  f  f i c  e  r ( s  ) 
p  ross  ope  r  a  t  o  '•  (  s  ; 
repa i rer ( s  ) 

service-member*  s  )  ;  serv  i  < 

men  and  serv  i  c<-vomen 
s  po ke  s  pe  r s  on 

f  ;  i  gh  t.  a  t  t  ondan  t  (  s  )  ;  ca  >  i  n 
at  t  endan t  ( s  ) 

unoe  c  up  i  ed  aim  ra  f  t.  ; 
pilotless  air  c r a f  t  ; 
remo  t.  e  1  y  p  i  1  o  t  ed 
vehicle 

unch-rs  t  af  f  ed 

vat  chg'uard  (  s  ) 

v i re  r ( s )  ;  v i revo r ke  r ( s ) 

worker’s  compensat  i  <  >  ri 


(estraited  from  APR  ~>  -  8  A  t  t  achmen  t  2,  1  9 1!  5  :  a  7  -  0  1  ) 


ADD  I T ION  XL  ORGAN  I Z AT  I ON S 
INVOLVED  IN  THE  DISTRIBUTION  OF  AECMA 
DOCUMENT  =  PSC -35-16598 


1.  National  Associations  who  are  members  of  AECMA,  and  all 

their  member  companies. 

2.  Members  of  Association  of  European  Airl ines  { AEA ) . 

Members  of  Aerospace  Industries  Association  of  America 
(  A  I A  )  . 

1.  Members  of  Air  Transport  Association  of  America  ( ATA ) . 

5.  Members  of  International  Coordinating’  Council  of 

Aerospace  Industries  Associations  (ICC'AIA),  not  i.ieludtd 
in  cat  eg'o  r  i  e  s  1  to  4  i  no  i  us  i  ve  . 

A.  Ministries  of  Defense  of  the  member  countries  of  AEC M v . 

7.  The  Department  cf  Defense  of  the  U.S.A. 

(extracted  from  AECMA,  1 963 : unnumbered  page) 


Appendix  C':  Computat  iona  1  Methodology 
fur  De  t  erm  i  n  I  in;-  the  RGL  of  Air  Force  TOs 


(extracted  from  MIL-M-38784B,  1983-38-40) 

4.4.1.  Sample  selection.  Samples  of  text  shall  be  analyzed 
for  readability.  Select  samples  as  follows: 

a.  Count  number  of  pages  of  text  in  publication.  The  count 
shall  include  all  full  and  partial  pag'es  that  contain  text 

i  t'i  the  form  of  consecutive  sentences.  The  conn,  shall  not 
include  pag'es  containing  only  illustrations,  tables,  lists, 
etc.  Record  number  of  text  pages. 

b.  The  basic  number  of  samples  is  determined  by  the 


f  o  1  1  o  -.x  i  n  g  : 

DIVISOR 

BASIC  NO. 

OF 

SAMPLES 

NO.  OF  TEXT  PAGES 

DIVIDE  BY  "N" 

M I N 

max 

90  and  above 

1  0 

9 

:tf) 

54  to  89 

9 

G 

9 

3  2  to  5 3 

8 

4 

6 

1  to  3  1 

G 

•> 

1 

c.  Divide  number  of  text  pages  by  appropriate  divisor, 
Round  off  quot  ient  t  o  the  next  invest  whole  number .  For 
example,  qiiot  ients  of  17  .id  17.7  shell  1  both  be  rounded 


off  to  1 

7  . 

This  quol.  ient  shal  i  equal  has 

ic  i  nimbi 

•r  of 

s  amp  1  e  rs 

t  o 

be  analyzed. 

d .  For 

pun 

1  i cat  ions  that  are  less  than  1 

2  pages, 

random i \ 

select  t 

A  1  ) 

samples  and  mark  t hem  for  anal 

yses  . 

/  r, 


For  pub  1  ieat  ions  that  are  12  pages  or  more, 


random  1 v 


select  a  number  between  one  and  "N."  The  number  selected 
shall  be  marked  as  first  pag'e  of  text  to  oe  analyzed. 
Starting'  at  selected  pag'e,  mark  every  "Nth"  page  of  text 
end  of  publication.  The  marked  pages  snail  identify 
starting  points  for  remaining'  basic  samples  to  be  analyze 
f.  Cheek  marked  pag'e  s  to  verify  ?  hat  at  least  one  sample 
has  been  selected  for  each  chapter  of  the  pub  1  i cat  i on .  I 
any  chapter  has  been  missed,  randomly  select  one  text  pag 
from  that  chapter  and  add  it  to  the  basic  sample.',  to  be 
ana  !  y  r, e d  . 


1.  1.2  Raw  data  collection.  For  each  sample  marked,  raw 
data  must  be  collected.  Data  collection  will  consist  of 
counts  of  the  numbers  of  words ,  sentences  and  syllables  i 
each  sample.  The  size  of  each  sample  is  based  on  the  n urn 
of  words  to  be  analyzed. 

4  .  4  .  2  .  1  Word  Count  .  Most  samples  will  be  slightly  mom 
than  200  words;  in  some  cases,  samples  will  be  smaller. 
Word  count  samples  shall  he  determined  as  follows: 
a.  Samples  shall  start  at  the  beginning'  of  the  first  ful 


paragraph  on  each  marked  sample  page.  If  sample  fa!  I-'  on 
pag'e  containing  procedural  instructions,  start  sample  at 
beginning  of  first  full  sentence  on  pag'e.  Headings, 
cap!  ions,  and  paragraph  t  it  les  shal  1  not  be  counted  in  t  h 
S amp  1 e . 


I;.  For  each  sample,  count  all  words  up  to  the  end  uf 
sentence  containing  the  200th  word.  If  the  marked  sample 
P'.age  is  less  than  200  words ,  sample  can  be  extended  to  next 
page  of  text;  but,  do  not  extend  sample  into  a  new  chapter 
or  text  pertaining  to  a  completely  nev  subject. 

c.  Count  as  a  word  all  numbers,  letters,  symbols,  and  gruu 
f  letters  surrounded  by  white  spaces.  Hyphens*  ed  vt'dt  a:. 

c>:  ft  t  rae  t  ions  count  as  one  word.  For  example,  each  of  the 
following  count  as  one  word:  couldn’t;  GFE;  i.e.;  .2, OOF; 

1 3- inch;  +  2  5  *  F . 

d.  Record  number  of  words  in  each  sample. 

,.-1.2.2  Sent  eno  e  c  oun  t  .  For  each  sample,  count  number  >  f 
sentences  as  follows: 

a.  Count  all  sentences  in  sample  including  sentence  that 
contains  the  200th  word. 

b .  Count  as  a  sentence  each  unit  of  thought  that,  can  he 
considered  grammatically  independent  of  another  sentence  or 
clause.  A  period,  question  mark,  exclamation  point,  and 
semi-colon  usually  denote  independent  clauses  and  thus  mark 
the  end  of  a  sentence. 

c .  Record  n  u m  o  w  *  >  >  f  s ■  ■  i  \  1 1  • i  i  c  e  s  in  each  samp  1 e . 

!  .1  .  2  .  f.  Sy  1  1  ab  1  e  coun  t  .  For  each  sample,  count  number  of 
syllables  as  follows: 

a.  For  most  words,  count  syllables  the  way  the  cord  is 
normal 1  \  pronounced  aloud.  For  example:  at  is  one 

syl iable,  maintain  is  two  syllables,  area  is  three 


syllable's,  panoramic  is  four  syllables,  arid  recuperator  is 
f i ve  sy 1  1 ab 1 es  . 

b.  Count  all  numbers  as  one  syllable.  For  example,  5.1, 

65,  300  all  count  as  one  syllable.  However,  if  a  numeric- 

expression  contains  several  numbers  separated  by  hyphens, 
count  each  number  as  a  syllable.  For  example,  i n  the 
expression  TM  9-1025-240-10,  9-1025-240-10  is  counted  us 
four  sy 1  tables. 

c.  Acronyms  and  abbreviations  are  counted  as  one  sy'  lub'.e 
unless  they  actual ly  spell  out  a  word  of  more  than  one 

sy!  [able.  For  example,  Hz  and  DAAl  each  count  as  one 
syllable  but  TRADOC  and  ARRC'OM  each  count  as  two  syllables. 

d.  Record  number  of  syllables  in  ecteh  sample. 

1.4.  3  Grade  I'-vel  calculat  ions. 

4.4.3.  1  Over  a  1  I grade  1  e  vel  .  The  Over  a  1  1  Grade  I. eve  1  t  <  ’GI.  j 

i.f  a  publication  is  calculated  as  follows: 

a.  Add  up  total  number  of  words  from  all  samples  combined. 
Re  cord  total. 

b.  Add  up  total  number  of  sentences  from  all  samples 
comb i n e d .  R e e o r d  total  . 

c.  Add  up  total  number  of  syllables  from  all  sampler 
combined.  Record  total . 

■  I.  Calculate  t.  he  average  sentence  length.  Divine  total 
number  of  words  by  total  number  of  sentences.  Round  off 
ijuot  ient  to  *  he  nearest  one-hundredth.  Record  quot  ient. 


t*  ♦ 


Ca  leu  I  at  i*  the  average  number  of  syllables  per  '.-or  1. 


Divide  total  number  of  syllables  by  total  number  of  -orb,  . 
Round  off  quotient  to  the  nearest  one -hundred t h .  Record 
quo  t  i en  t  . 

f.  Calculate  the  Overall  Grade  Level  ( OGL )  of  publication 
by  the  fei loving  formula: 

OGL- 0 . 39  (Avg  words / s en t ence ) -  1 1 . 3  (  \vg 

sy 1  1 ab 1 es /word ) -  1 5 . 50 

Round  off  the  OGL  t o  the  nearest  one- ; ent  n 
1  .  2  Sample  grade  levels.  fa  1  e  u  l  at  e  t  he  GL  of  each 

s  amp  1 e  a s  f  o 1 1 ovs : 

a.  Calculate  the  average  sentence  leng'th.  Di  %  i  de  tuuibe:'  1 
words  i n  sample  by  number  of  sentences  in  sample.  Round  of 
quotient  t  o  the  nearest  one-hundred t h . 

b .  Calculate  the  average  number  of  syllables  per  vur  :i . 
Divide  number  of  syllables  in  sample  by  number  of  words  in 
sample.  Round  off  quotient  to  the  nearest  one-hund red t h . 

c  .  Calculate  the-  Grade  Level  { GL )  of  each  sample  by  the 
f  o  1  1  ow  i  ng’  f  o  rmu  1  a  : 

G L  =  0 . 3 9  ( Avg  vords/sentences )  +  l  1 .8 
{  A  v  g  s  y 1  1  a b 1 e s / wo  r d ) -  1 o . o  u 
Round  off  each  Gb  to  the  nearest  one -tent 
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